mlsanityData to 14 Oct 2026

Are Multilingual Models Actually Improving? Isolating True Cross-Lingual Transfer

arXiv 2606.21954 ↗First posted 20 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-lingual transfer is a model's ability to generalize capabilities from well-represented source languages to under-represented target languages. Existing measures of a model's transfer strength conflate improvements in transfer with general improvements to accuracy in the source language. We advocate for an alternate metric that reliably captures transfer strength called Hardness Adjusted Transfer (HAT) Score, and use it to derive multiple insights on factors influencing transfer strength. Our analysis across twenty diverse language models and three popular mainstream multilingual benchmarks argues that 1) transfer in small models is not broken, 2) we are making slower than expected progress in cross-lingual transfer with model size, and 3) we have made clear progress over time.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 20 Jun 2026.

Builds on

24 of 41 references.

References · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  3. 2024A Survey on LLM-as-a-Judgecited by 1,423
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  6. 2019How multilingual is Multilingual BERT?cited by 645
  7. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  8. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  9. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  10. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  11. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  12. 2024How do Large Language Models Handle Multilingualism?cited by 150
12 more
  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 84
  2. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 54
  3. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  4. 2024Probing the Emergence of Cross-lingual Alignment during LLM Trainingcited by 39
  5. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 34
  6. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 29
  7. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 21
  8. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasonerscited by 16
  9. 2025MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languagescited by 15
  10. 2024ImageNot: A contrast with ImageNet preserves model rankingscited by 14
  11. 2025Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Resultscited by 10
  12. 2025Rethinking Cross-lingual Gaps from a Statistical Viewpointcited by 4

Built on by

Not cited yet.