mlsanityData to 14 Oct 2026

Morphology Matters: A Multilingual Language Modeling Analysis

arXiv 2012.06262 ↗First posted 11 Dec 2020cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Prior studies in multilingual language modeling (e.g., Cotterell et al., 2018; Mielke et al., 2019) disagree on whether or not inflectional morphology makes languages harder to model. We attempt to resolve the disagreement and extend those studies. We compile a larger corpus of 145 Bible translations in 92 languages and a larger number of typological features. We fill in missing typological data for several languages and consider corpus-based measures of morphological complexity in addition to expert-produced typological features. We find that several morphological measures are significantly associated with higher surprisal when LSTM models are trained with BPE-segmented data. We also investigate linguistically-motivated subword segmentation strategies like Morfessor and Finite-State Transducers (FSTs) and find that these segmentation strategies yield better performance and reduce the impact of a language's morphology on language modeling.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 2; 2023: 6; 2024: 7; 2025: 10; 2026: 7.

37 papers.

Sort:

By area:

  1. 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 108
  2. 2024Why do language models perform worse for morphologically complex languages?cited by 62
  3. 2024MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modelingcited by 40
  4. 2021You should evaluate your language model on marginal likelihood over tokenisationscited by 30
  5. 2023Analyzing Cognitive Plausibility of Subword Tokenizationcited by 30
  6. 2025MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologiescited by 25
  7. 2024Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5cited by 14
  8. 2025Confounding Factors in Relating Model Performance to Morphologycited by 14
  9. 2024Emergent Language: A Survey and Taxonomycited by 13
  10. 2023Effects of sub-word segmentation on performance of transformer language modelscited by 12
  11. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 11
  12. 2025Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Mattercited by 10
25 more
  1. 2021A Masked Segmental Language Model for Unsupervised Natural Language Segmentationcited by 9
  2. 2025TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairscited by 9
  3. 2021A multilabel approach to morphosyntactic probingcited by 8
  4. 2023Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillationcited by 6
  5. 2022Measuring Geographic Performance Disparities of Offensive Language Classifierscited by 5
  6. 2023Average Is Not Enough: Caveats of Multilingual Evaluationcited by 5
  7. 2022Morphological Processing of Low-Resource Languages: Where We Are and What's Nextcited by 4
  8. 2023Tokenization with Factorized Subword Encodingcited by 4
  9. 2024A Morphology-Based Investigation of Positional Encodingscited by 4
  10. 2025Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methodscited by 3
  11. 2026Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?cited by 3
  12. 2024Word Boundary Information Isn't Useful for Encoder Language Modelscited by 2
  13. 2024Recent advancements in computational morphology : A comprehensive surveycited by 2
  14. 2025Limitations of Religious Data and the Importance of the Target Domain: Towards Machine Translation for Guinea-Bissau Creolecited by 2
  15. 2026Form and Meaning in Intrinsic Multilingual Evaluationscited by 2
  16. 2026Tokenization with Split Treescited by 2
  17. 2021Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transductioncited by 1
  18. 2021Multilingual unsupervised sequence segmentation transfers to extremely low-resource languagescited by 1
  19. 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmeticcited by 1
  20. 2025On the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibilitycited by 1
  21. 2025Contextual morphologically-guided tokenization for Latin encoder modelscited by 1
  22. 2026Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Modelscited by 1
  23. 2025MoVoC: Morphology-Aware Subword Construction for Geez Script Languagescited by 0
  24. 2026ltzGLUE: Luxembourgish General Language Understanding Evaluationcited by 0
  25. 2026When Tokenization is Secretly Output Supervisioncited by 0