mlsanityData to 14 Oct 2026

Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Tasks

arXiv 2505.24834 ↗First posted 30 May 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-lingual transfer is central to modern NLP, enabling models to perform tasks in languages different from those they were trained on. A common assumption is that training on more languages improves zero-shot transfer. We test this on sense-aware tasks-polysemy and lexical semantic change-and find that multilinguality is not necessary for effective transfer. Our large-scale analysis across 28 languages reveals that other factors, such as differences in pretraining and fine-tuning data and evaluation artifacts, better explain the perceived benefits of multilinguality. We also release fine-tuned models and provide empirical baselines to support future research. While focused on two sense-aware tasks, our findings offer broader insights into cross-lingual transfer, especially for low-resource languages.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

31 of 57 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  5. 2019How multilingual is Multilingual BERT?cited by 645
  6. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  7. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  8. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  9. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  10. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  11. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  12. 2021MuRIL: Multilingual Representations for Indian Languagescited by 179
19 more
  1. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  2. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 108
  3. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  4. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  5. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  6. 2020XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualizationcited by 42
  7. 2020UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detectioncited by 32
  8. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 29
  9. 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
  10. 2021Predicting the Performance of Multilingual NLP Modelscited by 24
  11. 2024A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Changecited by 21
  12. 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
  13. 2021AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examplescited by 14
  14. 2018Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specializationcited by 12
  15. 2024Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Modelscited by 11
  16. 2023Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representationscited by 4
  17. 2025SenWiCh: Sense-Annotation of Low-Resource Languages for WiC using Hybrid Methodscited by 4
  18. 2023Self-Augmentation Improves Zero-Shot Cross-Lingual Transfercited by 3
  19. 2025LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Datacited by 2

Built on by

Not cited yet.