# A Reproducibility Study on Quantifying Language Similarity: The Impact of Missing Values in the URIEL Knowledge Base

arXiv: https://arxiv.org/abs/2405.11125 · HTML: https://mlsanity.com/paper/2405.11125/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 11 of 11)

- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://mlsanity.com/paper/1905.12688/) (2019)
- [UDapter: Language Adaptation for Truly Universal Dependency Parsing](https://mlsanity.com/paper/2004.14327/) (2020)
- [Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing](https://mlsanity.com/paper/1807.00914/) (2018)
- [A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies](https://mlsanity.com/paper/2301.01967/) (2023)
- [Predicting Performance for Natural Language Processing Tasks](https://mlsanity.com/paper/2005.00870/) (2020)
- [From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers](https://mlsanity.com/paper/2005.00633/) (2020)
- [Massively Multilingual Adversarial Speech Recognition](https://mlsanity.com/paper/1904.02210/) (2019)
- [Predicting the Performance of Multilingual NLP Models](https://mlsanity.com/paper/2110.08875/) (2021)
- [Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training](https://mlsanity.com/paper/2104.08645/) (2021)
- [LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization](https://mlsanity.com/paper/2401.06034/) (2024)

## Built on by (most cited first, 7 of 7)

- [URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base](https://mlsanity.com/paper/2409.18472/) (2024)
- [Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter](https://mlsanity.com/paper/2501.14491/) (2025)
- [data2lang2vec: Data Driven Typological Features Completion](https://mlsanity.com/paper/2409.17373/) (2024)
- [Shared Path: Unraveling Memorization in Multilingual LLMs through Language Similarities](https://mlsanity.com/paper/2505.15722/) (2025)
- [Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+](https://mlsanity.com/paper/2510.19217/) (2025)
- [Simple Additions, Substantial Gains: Expanding Scripts, Languages, and Lineage Coverage in URIEL+](https://mlsanity.com/paper/2510.27183/) (2025)
- [Multilinguality of Large Language Models From a Structural Perspective](https://mlsanity.com/paper/2606.01800/) (2026)

## Abstract (arXiv)

In the pursuit of supporting more languages around the world, tools that characterize properties of languages play a key role in expanding the existing multilingual NLP research. In this study, we focus on a widely used typological knowledge base, URIEL, which aggregates linguistic information into numeric vectors. Specifically, we delve into the soundness and reproducibility of the approach taken by URIEL in quantifying language similarity. Our analysis reveals URIEL's ambiguity in calculating language distances and in handling missing values. Moreover, we find that URIEL does not provide any information about typological features for 31% of the languages it represents, undermining the reliabilility of the database, particularly on low-resource languages. Our literature review suggests URIEL and lang2vec are used in papers on diverse NLP tasks, which motivates us to rigorously verify the database as the effectiveness of these works depends on the reliability of the information the tool provides.