mlsanityData to 14 Oct 2026

Are All Languages Equally Hard to Language-Model?

arXiv 1806.03743 ↗First posted 10 Jun 2018cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on languages with differing typological profiles? In this work, we develop an evaluation framework for fair cross-linguistic comparison of language models, using translated text so that all models are asked to predict approximately the same information. We then conduct a study on 21 languages, demonstrating that in some languages, the textual expression of the information is harder to predict with both n-gram and LSTM language models. We show complex inflectional morphology to be a cause of performance differences among languages.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 5; 2023: 9; 2024: 8; 2025: 10; 2026: 13.

53 papers.

Sort:

By area:

  1. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  2. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 132
  3. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  4. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
  5. 2024Why do language models perform worse for morphologically complex languages?cited by 62
  6. 2021Examining the Inductive Bias of Neural Language Models with Artificial Languagescited by 40
  7. 2024MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modelingcited by 40
  8. 2023Dissociating language and thought in large language modelscited by 29
  9. 2023Zero-shot cross-lingual transfer language selection using linguistic similaritycited by 28
  10. 2025Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMscited by 20
  11. 2024Evaluating Morphological Compositional Generalization in Large Language Modelscited by 19
  12. 2024gzip Predicts Data-dependent Scaling Lawscited by 16
38 more
  1. 2024Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5cited by 14
  2. 2025Confounding Factors in Relating Model Performance to Morphologycited by 14
  3. 2022Estimating the Entropy of Linguistic Distributionscited by 12
  4. 2021A Cognitive Regularizer for Language Modelingcited by 11
  5. 2022DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languagescited by 11
  6. 2024Emergent Word Order Universals from Cognitively-Motivated Language Modelscited by 11
  7. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 11
  8. 2023Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distributioncited by 10
  9. 2023MorphPiece : A Linguistic Tokenizer for Large Language Modelscited by 9
  10. 2025Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignmentcited by 9
  11. 2021On the Difficulty of Translating Free-Order Case-Marking Languagescited by 8
  12. 2022Transfer Language Selection for Zero-Shot Cross-Lingual Abusive Language Detectioncited by 8
  13. 2023GPT-SW3: An Autoregressive Language Model for the Nordic Languagescited by 8
  14. 2025Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attentioncited by 8
  15. 2021Modeling the Unigram Distributioncited by 7
  16. 2021The interplay between language similarity and script on a novel multi-layer Algerian dialect corpuscited by 5
  17. 2021Using AntiPatterns to avoid MLOps Mistakescited by 5
  18. 2025Which Word Orders Facilitate Length Generalization in LMs? An Investigation with GCG-Based Artificial Languagescited by 4
  19. 2021Towards Zero-shot Language Modelingcited by 3
  20. 2024Machine Translation Models are Zero-Shot Detectors of Translation Directioncited by 3
  21. 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamicscited by 3
  22. 2026Apples to Apples? Towards Comparable Crosslingual Language Model Evaluationcited by 3
  23. 2025Limitations of Religious Data and the Importance of the Target Domain: Towards Machine Translation for Guinea-Bissau Creolecited by 2
  24. 2025Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translationscited by 2
  25. 2026Form and Meaning in Intrinsic Multilingual Evaluationscited by 2
  26. 2023On the N-gram Approximation of Pre-trained Language Modelscited by 1
  27. 2024Morphological Typology in BPE Subword Productivity and Language Modelingcited by 1
  28. 2025Graph-Linguistic Fusion: Using Language Models for Wikidata Vandalism Detectioncited by 1
  29. 2025Rethinking Memorization Measures and their Implications in Large Language Modelscited by 1
  30. 2026Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Modelscited by 1
  31. 2026Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspectivecited by 1
  32. 2026Implicit Representations of Grammaticality in Language Modelscited by 1
  33. 2023Still no evidence for an effect of the proportion of non-native speakers on language complexity -- A response to Kauhanen, Einhaus & Walkden (2023)cited by 0
  34. 2023Curricular Transfer Learning for Sentence Encoded Taskscited by 0
  35. 2025MoVoC: Morphology-Aware Subword Construction for Geez Script Languagescited by 0
  36. 2026Vocabulary shapes cross-lingual variation of word-order learnability in language modelscited by 0
  37. 2026What Kind of Language is Easy to Language-Model Under Curriculum Learning?cited by 0
  38. 2026Brain-LLM Alignment Tracks Training Data, Not Typologycited by 0

50 most cited of 53 All 53 (JSON)