Language Distances are Practical for Equitable Cross-Lingual Transfer
Not on the map. Why?
Abstract (arXiv)
Cross-lingual transfer is strongly conditional on how the source language is chosen, but it is impractical to determine the best candidate source for every target language, especially for low-resource target languages. Language distances are widely used to rank candidate sources due to their correlation with transfer efficacy and applicability in resource-sparse settings. However, the reliability of distance-based rankers across tasks and resource levels remains underexplored. We therefore present the first equity-focused evaluation of paradigms for ranking source languages, studying resource-level inequality and task inequality across ten cross-lingual tasks and two multilingual models. While both inequalities are most pronounced for individual language distances and an English-always baseline, they are substantially reduced by training-free composite distances, and nearly eliminated by trained rankers. We further demonstrate the reliability of rankers using language distances compared to rankers using language model internals. Overall, we find that language distances provide a practical basis for equitable and performant transfer language selection. We recommend using trained rankers when task-specific transfer evaluations are available, and composite distances otherwise.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 67 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
40 of 67 references.
References · 40
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
- 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
- 2019How multilingual is Multilingual BERT?cited by 645
- 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
- 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
- 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
- 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
- 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
28 more
- 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
- 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
- 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
- 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
- 2024Conformal Prediction for Natural Language Processing: A Surveycited by 53
- 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
- 2020Predicting Performance for Natural Language Processing Taskscited by 49
- 2021Revisiting the Primacy of English in Zero-shot Cross-lingual Transfercited by 47
- 2022A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bankcited by 29
- 2023Zero-shot cross-lingual transfer language selection using linguistic similaritycited by 28
- 2023Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarizationcited by 28
- 2021Predicting the Performance of Multilingual NLP Modelscited by 24
- 2024URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Basecited by 21
- 2024ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Modelscited by 20
- 2022On the Calibration of Massively Multilingual Language Modelscited by 19
- 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
- 2023Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Spacecited by 12
- 2024LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalizationcited by 10
- 2025Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Mattercited by 10
- 2020Model Selection for Cross-Lingual Transfercited by 9
- 2025Untangling the Influence of Typology, Data and Model Architecture on Ranking Transfer Languages for Cross-Lingual POS Taggingcited by 9
- 2024Two-stage Risk Control with Application to Ranked Retrievalcited by 6
- 2024Analysis of Multi-Source Language Training in Cross-Lingual Transfercited by 5
- 2025Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfercited by 3
- 2025Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+cited by 3
- 2024Streamlining Conformal Information Retrieval via Score Refinementcited by 2
- 2025Transductive Conformal Inference for Full Rankingcited by 2
- 2025Simple Additions, Substantial Gains: Expanding Scripts, Languages, and Lineage Coverage in URIEL+cited by 2
Built on by
Not cited yet.