mlsanityData to 7 Oct 2026

A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank

arXiv 2205.04086 ↗First posted 9 May 2022cs.CL cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

We show that the choice of pretraining languages affects downstream cross-lingual transfer for BERT-based models. We inspect zero-shot performance in balanced data conditions to mitigate data size confounds, classifying pretraining languages that improve downstream performance as donors, and languages that are improved in zero-shot performance as recipients. We develop a method of quadratic time complexity in the number of languages to estimate these relations, instead of an exponential exhaustive computation of all possible combinations. We find that our method is effective on a diverse set of languages spanning different linguistic features and two downstream tasks. Our findings can inform developers of large-scale multilingual language models in choosing better pretraining configurations.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

15 of 29 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2018Deep contextualized word representationscited by 2,285
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  7. 2019How multilingual is Multilingual BERT?cited by 642
  8. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  9. 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
  10. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  11. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 180
  12. 2021Revisiting the Primacy of English in Zero-shot Cross-lingual Transfercited by 47
3 more
  1. 2022Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studiescited by 9
  2. 2021Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approachcited by 8
  3. 2021Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 7; 2025: 6; 2026: 9.

29 papers.

Sort:

By area:

  1. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  2. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  3. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 71
  4. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  5. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  6. 2024MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modelingcited by 40
  7. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 28
  8. 2024Goldfish: Monolingual Language Models for 350 Languagescited by 27
  9. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
  10. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
  11. 2025Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Mattercited by 10
  12. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 10
17 more
  1. 2024An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Modelscited by 7
  2. 2022You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Modelscited by 6
  3. 2023GradSim: Gradient-Based Language Grouping for Effective Multilingual Trainingcited by 6
  4. 2025Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuningcited by 4
  5. 2023Analysing Cross-Lingual Transfer in Low-Resourced African Named Entity Recognitioncited by 2
  6. 2025LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMscited by 2
  7. 2022Language Varieties of Italy: Technology Challenges and Opportunitiescited by 1
  8. 2026Skill Issue: Are Skills Language-Invariant in LLMs?cited by 1
  9. 2024MaiNLP at SemEval-2024 Task 1: Analyzing Source Language Selection in Cross-Lingual Textual Relatednesscited by 0
  10. 2025MiniLingua: A Small Open-Source LLM for European Languagescited by 0
  11. 2026Dialects of Translationese Shape Language Model Learningcited by 0
  12. 2026Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMscited by 0
  13. 2026Why Low-Resource NLP Needs More Than Cross-Lingual Transfer: Lessons Learned from Luxembourgishcited by 0
  14. 2026An In-Vitro Study on Cross-Lingual Generalization in Language Modelscited by 0
  15. 2026When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learningcited by 0
  16. 2026Which Languages Transfer Best to Warlpiri? A Similarity-Based Study for Low-Resource ASRcited by 0
  17. 2026Language Distances are Practical for Equitable Cross-Lingual Transfercited by 0