# Zero-shot cross-lingual transfer language selection using linguistic similarity

arXiv: https://arxiv.org/abs/2301.13720 · HTML: https://mlsanity.com/paper/2301.13720/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 95 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://mlsanity.com/paper/2004.01401/) (2020)
- [Cross-Lingual Ability of Multilingual BERT: An Empirical Study](https://mlsanity.com/paper/1912.07840/) (2019)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation](https://mlsanity.com/paper/2104.07412/) (2021)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://mlsanity.com/paper/1905.12688/) (2019)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [75 Languages, 1 Model: Parsing Universal Dependencies Universally](https://mlsanity.com/paper/1904.02099/) (2019)
- [Zero-Shot Cross-Lingual Transfer with Meta Learning](https://mlsanity.com/paper/2003.02739/) (2020)
- [Few-shot classification in Named Entity Recognition Task](https://mlsanity.com/paper/1812.06158/) (2018)
- [Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing](https://mlsanity.com/paper/1807.00914/) (2018)
- [Are All Languages Equally Hard to Language-Model?](https://mlsanity.com/paper/1806.03743/) (2018)
- [Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer](https://mlsanity.com/paper/2106.16171/) (2021)
- [Learning Language Representations for Typology Prediction](https://mlsanity.com/paper/1707.09569/) (2017)
- [Entity Projection via Machine Translation for Cross-Lingual NER](https://mlsanity.com/paper/1909.05356/) (2019)
- [Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi](https://mlsanity.com/paper/2109.03552/) (2021)
- [Cross-lingual, Character-Level Neural Morphological Tagging](https://mlsanity.com/paper/1708.09157/) (2017)

## Built on by (most cited first, 28 of 28)

- [LLaMA Beyond English: An Empirical Study on Language Capability Transfer](https://mlsanity.com/paper/2401.01055/) (2024)
- [Lost in Translation: Large Language Models in Non-English Content Analysis](https://mlsanity.com/paper/2306.07377/) (2023)
- [Quantifying the Dialect Gap and its Correlates Across Languages](https://mlsanity.com/paper/2310.15135/) (2023)
- [When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages](https://mlsanity.com/paper/2311.09205/) (2023)
- [Goldfish: Monolingual Language Models for 350 Languages](https://mlsanity.com/paper/2408.10441/) (2024)
- [Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models](https://mlsanity.com/paper/2311.09194/) (2023)
- [Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale](https://mlsanity.com/paper/2407.02118/) (2024)
- [Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners](https://mlsanity.com/paper/2405.13816/) (2024)
- [Untangling the Influence of Typology, Data and Model Architecture on Ranking Transfer Languages for Cross-Lingual POS Tagging](https://mlsanity.com/paper/2503.19979/) (2025)
- [Cyberbullying Detection for Low-resource Languages and Dialects: Review of the State of the Art](https://mlsanity.com/paper/2308.15745/) (2023)
- [How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective](https://mlsanity.com/paper/2505.21505/) (2025)
- [FilBench: Can LLMs Understand and Generate Filipino?](https://mlsanity.com/paper/2508.03523/) (2025)
- [Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+](https://mlsanity.com/paper/2510.19217/) (2025)
- [Multilingual Synopses of Movie Narratives: A Dataset for Vision-Language Story Understanding](https://mlsanity.com/paper/2406.13092/) (2024)
- [PortBERT: Navigating the Depths of Portuguese Language Models](https://mlsanity.com/paper/2606.02100/) (2026)
- [Improving Polish to English Neural Machine Translation with Transfer Learning: Effects of Data Volume and Language Similarity](https://mlsanity.com/paper/2306.00660/) (2023)
- [LegalTurk Optimized BERT for Multi-Label Text Classification and NER](https://mlsanity.com/paper/2407.00648/) (2024)
- [Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective](https://mlsanity.com/paper/2410.07239/) (2024)
- [From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models](https://mlsanity.com/paper/2602.09826/) (2026)
- [Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR](https://mlsanity.com/paper/2609.15758/) (2026)
- [Can MLLMs Generalize to Multi-Party dialog? Exploring Multilingual Response Generation in Complex Scenarios](https://mlsanity.com/paper/2501.11269/) (2025)
- [Using Machine Learning to Enhance the Detection of Obfuscated Abusive Words in Swahili: A Focus on Child Safety](https://mlsanity.com/paper/2602.13455/) (2026)
- [Budget-Xfer: Budget-Constrained Source Language Selection for Cross-Lingual Transfer to African Languages](https://mlsanity.com/paper/2603.27651/) (2026)
- [Harnessing Linguistic Dissimilarity for Language Generalization on Unseen Low-Resource Varieties](https://mlsanity.com/paper/2605.04500/) (2026)
- [Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer](https://mlsanity.com/paper/2606.19346/) (2026)
- [Evaluating the Effect of Linguistic Relatedness on Cross-Lingual Transfer in Large Multilingual Automatic Speech Recognition](https://mlsanity.com/paper/2607.04814/) (2026)
- [Language Distances are Practical for Equitable Cross-Lingual Transfer](https://mlsanity.com/paper/2609.32331/) (2026)
- [RunyaNER: Auxiliary Language Selection for Runyankore NER](https://mlsanity.com/paper/2609.37543/) (2026)

## Abstract (arXiv)

We study the selection of transfer languages for different Natural Language Processing tasks, specifically sentiment analysis, named entity recognition and dependency parsing. In order to select an optimal transfer language, we propose to utilize different linguistic similarity metrics to measure the distance between languages and make the choice of transfer language based on this information instead of relying on intuition. We demonstrate that linguistic similarity correlates with cross-lingual transfer performance for all of the proposed tasks. We also show that there is a statistically significant difference in choosing the optimal language as the transfer source instead of English. This allows us to select a more suitable transfer language which can be used to better leverage knowledge from high-resource languages in order to improve the performance of language applications lacking data. For the study, we used datasets from eight different languages from three language families.