# From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

arXiv: https://arxiv.org/abs/1802.09375 · HTML: https://mlsanity.com/paper/1802.09375/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2018.

## Built on by (most cited first, 13 of 13)

- [Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture](https://mlsanity.com/paper/2407.07342/) (2024)
- [Does Typological Blinding Impede Cross-Lingual Sharing?](https://mlsanity.com/paper/2101.11888/) (2021)
- [Language Embeddings Sometimes Contain Typological Generalizations](https://mlsanity.com/paper/2301.08115/) (2023)
- [Multilingual Gradient Word-Order Typology from Universal Dependencies](https://mlsanity.com/paper/2402.01513/) (2024)
- [Language Embeddings for Typology and Cross-lingual Transfer Learning](https://mlsanity.com/paper/2106.02082/) (2021)
- [Morphological Inflection: A Reality Check](https://mlsanity.com/paper/2305.15637/) (2023)
- [Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectiveness](https://mlsanity.com/paper/2306.02646/) (2023)
- [LangSAMP: Language-Script Aware Multilingual Pretraining](https://mlsanity.com/paper/2409.18199/) (2024)
- [Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings](https://mlsanity.com/paper/2101.10565/) (2021)
- [Informative Language Representation Learning for Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/2209.01530/) (2022)
- [Do Multi-Lingual Pre-trained Language Models Reveal Consistent Token Attributions in Different Languages?](https://mlsanity.com/paper/2112.12356/) (2021)
- [mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2305.13684/) (2023)
- [Patterns of Persistence and Diffusibility across the World's Languages](https://mlsanity.com/paper/2401.01698/) (2024)

## Abstract (arXiv)

A core part of linguistic typology is the classification of languages according to linguistic properties, such as those detailed in the World Atlas of Language Structure (WALS). Doing this manually is prohibitively time-consuming, which is in part evidenced by the fact that only 100 out of over 7,000 languages spoken in the world are fully covered in WALS. We learn distributed language representations, which can be used to predict typological properties on a massively multilingual scale. Additionally, quantitative and qualitative analyses of these language embeddings can tell us how language similarities are encoded in NLP models for tasks at different typological levels. The representations are learned in an unsupervised manner alongside tasks at three typological levels: phonology (grapheme-to-phoneme prediction, and phoneme reconstruction), morphology (morphological inflection), and syntax (part-of-speech tagging). We consider more than 800 languages and find significant differences in the language representations encoded, depending on the target task. For instance, although Norwegian Bokmaal and Danish are typologically close to one another, they are phonologically distant, which is reflected in their language embeddings growing relatively distant in a phonological task. We are also able to predict typological features in WALS with high accuracies, even for unseen language families.