# Learning Language Representations for Typology Prediction

arXiv: https://arxiv.org/abs/1707.09569 · HTML: https://mlsanity.com/paper/1707.09569/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2017.

## Built on by (most cited first, 30 of 40)

- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models](https://mlsanity.com/paper/2204.08110/) (2022)
- [Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models](https://mlsanity.com/paper/2205.11758/) (2022)
- [Zero-shot cross-lingual transfer language selection using linguistic similarity](https://mlsanity.com/paper/2301.13720/) (2023)
- [Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?](https://mlsanity.com/paper/2409.19151/) (2024)
- [InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning](https://mlsanity.com/paper/2305.13627/) (2023)
- [Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features](https://mlsanity.com/paper/2310.06458/) (2023)
- [Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages](https://mlsanity.com/paper/2212.09651/) (2022)
- [Does Typological Blinding Impede Cross-Lingual Sharing?](https://mlsanity.com/paper/2101.11888/) (2021)
- [Language Embeddings Sometimes Contain Typological Generalizations](https://mlsanity.com/paper/2301.08115/) (2023)
- [Multilingual Gradient Word-Order Typology from Universal Dependencies](https://mlsanity.com/paper/2402.01513/) (2024)
- [Language Embeddings for Typology and Cross-lingual Transfer Learning](https://mlsanity.com/paper/2106.02082/) (2021)
- [A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space](https://mlsanity.com/paper/2109.06324/) (2021)
- [Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs](https://mlsanity.com/paper/2305.17740/) (2023)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [The Past, Present, and Future of Typological Databases in NLP](https://mlsanity.com/paper/2310.13440/) (2023)
- [Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis](https://mlsanity.com/paper/2410.13237/) (2024)
- [Polyglot Prompt: Multilingual Multitask PrompTraining](https://mlsanity.com/paper/2204.14264/) (2022)
- [Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations](https://mlsanity.com/paper/2404.02452/) (2024)
- [No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement](https://mlsanity.com/paper/2404.15737/) (2024)
- [Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages](https://mlsanity.com/paper/2507.03003/) (2025)
- [Meta-XNLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation](https://mlsanity.com/paper/2203.10250/) (2022)
- [UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages](https://mlsanity.com/paper/2406.09717/) (2024)
- [Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages](https://mlsanity.com/paper/2103.13272/) (2021)
- [Towards a Deep Understanding of Multilingual End-to-End Speech Translation](https://mlsanity.com/paper/2310.20456/) (2023)
- [Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectiveness](https://mlsanity.com/paper/2306.02646/) (2023)
- [Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models](https://mlsanity.com/paper/2402.18397/) (2024)
- [LangSAMP: Language-Script Aware Multilingual Pretraining](https://mlsanity.com/paper/2409.18199/) (2024)
- [M-Wanda: Improving One-Shot Pruning for Multilingual LLMs](https://mlsanity.com/paper/2505.21171/) (2025)

## Abstract (arXiv)

One central mystery of neural NLP is what neural models "know" about their subject matter. When a neural machine translation system learns to translate from one language to another, does it learn the syntax or semantics of the languages? Can this knowledge be extracted from the system to fill holes in human scientific knowledge? Existing typological databases contain relatively full feature specifications for only a few hundred languages. Exploiting the existence of parallel texts in more than a thousand languages, we build a massive many-to-one neural machine translation (NMT) system from 1017 languages into English, and use this to predict information missing from typological databases. Experiments show that the proposed method is able to infer not only syntactic, but also phonological and phonetic inventory features, and improves over a baseline that has access to information about the languages' geographic and phylogenetic neighbors.