mlsanityData to 14 Oct 2026

Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer

arXiv 2606.19346 ↗First posted 26 Apr 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We study cross-lingual transfer by fine-tuning seven large language models (4B--671B parameters) on Arabic and evaluating zero-shot reading comprehension on Semitic languages and non-Semitic controls. Across dense and Mixture-of-Experts architectures, we find no evidence of Semitic-specific transfer: models with weak baselines improve dramatically across all languages, while strong-baseline models show only marginal gains regardless of language family. A chain-of-thought ablation reinforces this finding -- the same models that benefit most from fine-tuning benefit equally from inference-time reasoning, suggesting both mechanisms address task-format alignment rather than cross-lingual knowledge transfer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

25 of 39 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  9. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  10. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  11. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  12. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
13 more
  1. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  2. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  3. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  4. 2023Mitigating the Alignment Tax of RLHFcited by 183
  5. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
  6. 2023LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugincited by 82
  7. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43
  8. 2020English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Toocited by 39
  9. 2020From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformerscited by 35
  10. 2023Zero-shot cross-lingual transfer language selection using linguistic similaritycited by 28
  11. 2024Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reductioncited by 18
  12. 2025Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphologycited by 3
  13. 2020Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs
  2. 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
  3. 2023Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers
  4. 2025Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs
  5. 2024Revisiting the Superficial Alignment Hypothesis
  6. 2025Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent?
  7. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs
  8. 2025Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World