mlsanityData to 14 Oct 2026

There is No Theoretical Curse of Multilinguality For Embedding Space Structure

arXiv 2608.17088 ↗First posted 17 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model. The curse of multilinguality describes the phenomenon of degradation in multilingual model performance as we increase language coverage, posing a threat to the above goal. This paper asks whether multilingual embedding spaces are inherently incapable of achieving perfect multilinguality without a prohibitive increase in required capacity. We first formalize the goal of "perfect multilinguality", embodied in two multilinguality conditions. We then prove that the minimum dimensionality required for perfect multilinguality grows only logarithmically in the number of languages. That is, we show that there is no theoretical curse of multilinguality for embedding space structure. This suggests that the empirical curse of multilinguality is a result of real world data and training conditions. We back this understanding with a small-scale empirical study. Our paper provides the first theoretical and intrinsic perspective on the curse of multilinguality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

33 of 43 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  4. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  6. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  7. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  8. 2019Intrinsic dimension of data representations in deep neural networkscited by 282
  9. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  10. 2019Massively Multilingual Neural Machine Translationcited by 246
  11. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  12. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 169
21 more
  1. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 146
  2. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  3. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 104
  4. 2022The Geometry of Multilingual Language Model Representationscited by 95
  5. 2020Balancing Training for Multilingual Neural Machine Translationcited by 72
  6. 2020LAReQA: Language-agnostic answer retrieval from a multilingual poolcited by 42
  7. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  8. 2022Causes and Cures for Interference in Multilingual Translationcited by 23
  9. 2025BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translationcited by 22
  10. 2025ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilingualitycited by 22
  11. 2023Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddingscited by 19
  12. 2024Discovering Low-rank Subspaces for Language-agnostic Multilingual Representationscited by 19
  13. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  14. 2024Exploring Alignment in Shared Cross-lingual Spacescited by 13
  15. 2024Multilingual Large Language Models and Curse of Multilingualitycited by 13
  16. 2025Revisiting Multilingual Data Mixtures in Language Model Pretrainingcited by 10
  17. 2023Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalizationcited by 6
  18. 2026Omnilingual MT: Machine Translation for 1,600 Languagescited by 5
  19. 2026What Limits Does Quantization Place on Dense Top-k Retrieval? A Theoretical Studycited by 4
  20. 2023The Geometry of Multilingual Language Models: An Equality Lenscited by 2
  21. 2026The Geometry of Forgettingcited by 2

Built on by

Not cited yet.