# Why do language models perform worse for morphologically complex languages?

arXiv: https://arxiv.org/abs/2411.14198 · HTML: https://mlsanity.com/paper/2411.14198/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 103 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 30 of 34)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://mlsanity.com/paper/2005.00333/) (2020)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects](https://mlsanity.com/paper/2309.07445/) (2023)
- [Systematic Inequalities in Language Technology Performance across the World's Languages](https://mlsanity.com/paper/2110.06733/) (2021)
- [XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models](https://mlsanity.com/paper/2301.10472/) (2023)
- [Getting the most out of your tokenizer for pre-training and domain adaptation](https://mlsanity.com/paper/2402.01035/) (2024)
- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://mlsanity.com/paper/2101.00403/) (2021)
- [Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance](https://mlsanity.com/paper/2403.06265/) (2024)
- [Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World](https://mlsanity.com/paper/2210.08523/) (2022)
- [Are All Languages Equally Hard to Language-Model?](https://mlsanity.com/paper/1806.03743/) (2018)
- [UniMorph 2.0: Universal Morphology](https://mlsanity.com/paper/1810.11101/) (2018)
- [SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2006.11572/) (2020)
- [Morphology Matters: A Multilingual Language Modeling Analysis](https://mlsanity.com/paper/2012.06262/) (2020)
- [Greed is All You Need: An Evaluation of Tokenizer Inference Methods](https://mlsanity.com/paper/2403.01289/) (2024)
- [Goldfish: Monolingual Language Models for 350 Languages](https://mlsanity.com/paper/2408.10441/) (2024)
- [A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages](https://mlsanity.com/paper/2403.00686/) (2024)
- [Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding](https://mlsanity.com/paper/2004.14870/) (2020)
- [Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages](https://mlsanity.com/paper/2205.12676/) (2022)
- [Morphological and Language-Agnostic Word Segmentation for NMT](https://mlsanity.com/paper/1806.05482/) (2018)
- [The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2103.11189/) (2021)
- [Lexically Grounded Subword Segmentation](https://mlsanity.com/paper/2406.13560/) (2024)
- [CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models](https://mlsanity.com/paper/2305.14214/) (2023)

## Built on by (most cited first, 30 of 62)

- [Evaluating Morphological Alignment of Tokenizers in 70 Languages](https://mlsanity.com/paper/2507.06378/) (2025)
- [ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality](https://mlsanity.com/paper/2510.22037/) (2025)
- [Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs](https://mlsanity.com/paper/2502.18795/) (2025)
- [Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations](https://mlsanity.com/paper/2506.19004/) (2025)
- [Confounding Factors in Relating Model Performance to Morphology](https://mlsanity.com/paper/2511.01380/) (2025)
- [Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts](https://mlsanity.com/paper/2505.22582/) (2025)
- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs](https://mlsanity.com/paper/2506.13487/) (2025)
- [Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment](https://mlsanity.com/paper/2508.08424/) (2025)
- [MorphTok: Morphologically Grounded Tokenization for Indian Languages](https://mlsanity.com/paper/2504.10335/) (2025)
- [Explaining and Mitigating Crosslingual Tokenizer Inequities](https://mlsanity.com/paper/2510.21909/) (2025)
- [Subword models struggle with word learning, but surprisal hides it](https://mlsanity.com/paper/2502.12835/) (2025)
- [Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models](https://mlsanity.com/paper/2601.13260/) (2026)
- [The Impact of Token Granularity on the Predictive Power of Language Model Surprisal](https://mlsanity.com/paper/2412.11940/) (2024)
- [Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMs](https://mlsanity.com/paper/2505.16134/) (2025)
- [MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment](https://mlsanity.com/paper/2606.27019/) (2026)
- [Which Word Orders Facilitate Length Generalization in LMs? An Investigation with GCG-Based Artificial Languages](https://mlsanity.com/paper/2510.12722/) (2025)
- [DetectRL-X: Towards Reliable Multilingual and Real-World LLM-Generated Text Detection](https://mlsanity.com/paper/2605.15518/) (2026)
- [UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?](https://mlsanity.com/paper/2508.11260/) (2025)
- [Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings](https://mlsanity.com/paper/2509.14405/) (2025)
- [Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossible](https://mlsanity.com/paper/2510.07178/) (2025)
- [Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish](https://mlsanity.com/paper/2602.03484/) (2026)
- [Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics](https://mlsanity.com/paper/2606.06533/) (2026)
- [Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation](https://mlsanity.com/paper/2608.25089/) (2026)
- [Limitations of Religious Data and the Importance of the Target Domain: Towards Machine Translation for Guinea-Bissau Creole](https://mlsanity.com/paper/2504.02674/) (2025)
- [Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translations](https://mlsanity.com/paper/2504.07680/) (2025)
- [Multilingual Tokenization through the Lens of Indian Languages: Challenges and Insights](https://mlsanity.com/paper/2506.17789/) (2025)
- [SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?](https://mlsanity.com/paper/2508.12243/) (2025)
- [Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pre-trained Models](https://mlsanity.com/paper/2512.03989/) (2025)
- [Form and Meaning in Intrinsic Multilingual Evaluations](https://mlsanity.com/paper/2601.10580/) (2026)

## Abstract (arXiv)

Language models perform differently across languages. It has been previously suggested that morphological typology may explain some of this variability (Cotterell et al., 2018). We replicate previous analyses and find additional new evidence for a performance gap between agglutinative and fusional languages, where fusional languages, such as English, tend to have better language modeling performance than morphologically more complex languages like Turkish. We then propose and test three possible causes for this performance gap: morphological alignment of tokenizers, tokenization quality, and disparities in dataset sizes and measurement. To test the morphological alignment hypothesis, we present MorphScore, a tokenizer evaluation metric, and supporting datasets for 22 languages. We find some evidence that tokenization quality explains the performance gap, but none for the role of morphological alignment. Instead we find that the performance gap is most reduced when training datasets are of equivalent size across language types, but only when scaled according to the so-called "byte-premium" -- the different encoding efficiencies of different languages and orthographies. These results suggest that no language is harder or easier for a language model to learn on the basis of its morphological typology. Differences in performance can be attributed to disparities in dataset size. These results bear on ongoing efforts to improve performance for low-performing and under-resourced languages.