mlsanityData to 7 Oct 2026

Multilinguality of Large Language Models From a Structural Perspective

arXiv 2606.01800 ↗First posted 1 Jun 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have excelled in processing multiple languages through pre- and post-training on multilingual data, even though English dominates the training data. Prior work focusing on token representations has revealed how those LLMs process non-English text. Although these analyses have provided insightful findings, they fail to capture a structural view, which is an inherent property of language. In this study, we explore the multilinguality of LLMs through representational structural analysis. Our findings reveal that low-resource languages are structurally more different from English than high- and mid-resource languages, and that language-specific post-training alters their structures while preserving inter-language relationships.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

18 of 38 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  3. 2020On Layer Normalization in the Transformer Architecturecited by 906
  4. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  5. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  6. 2022The Geometry of Multilingual Language Model Representationscited by 94
  7. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  8. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  9. 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
  10. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 57
  11. 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 35
  12. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
6 more
  1. 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
  2. 2024URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Basecited by 21
  3. 2024A Reproducibility Study on Quantifying Language Similarity: The Impact of Missing Values in the URIEL Knowledge Basecited by 7
  4. 2025Languages Still Left Behind: Toward a Better Multilingual Machine Translation Benchmarkcited by 7
  5. 2025Diversity of Transformer Layers: One Aspect of Parameter Scaling Lawscited by 2
  6. 2026StructLens: A Structural Lens for Language Models via Maximum Spanning Treescited by 1

Built on by

Not cited yet.