The Morphemic Origin of Zipf's Law: A Factorized Combinatorial Framework
Not on the map. Why?
Abstract (arXiv)
We present a simple structure based model of how words are formed from morphemes. The model explains two major empirical facts: the typical distribution of word lengths and the appearance of Zipf like rank frequency curves. In contrast to classical explanations based on random text or communication efficiency, our approach uses only the combinatorial organization of prefixes, roots, suffixes and inflections. In this Morphemic Combinatorial Word Model, a word is created by activating several positional slots. Each slot turns on with a certain probability and selects one morpheme from its inventory. Morphemes are treated as stable building blocks that regularly appear in word formation and have characteristic positions. This mechanism produces realistic word length patterns with a concentrated middle zone and a thin long tail, closely matching real languages. Simulations with synthetic morpheme inventories also generate rank frequency curves with Zipf like exponents around 1.1-1.4, similar to English, Russian and Romance languages. The key result is that Zipf like behavior can emerge without meaning, communication pressure or optimization principles. The internal structure of morphology alone, combined with probabilistic activation of slots, is sufficient to create the robust statistical patterns observed across languages.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 26 references that are in the map. First posted on arXiv 13 Dec 2025.
Builds on
6 of 26 references.
References · 6
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
- 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 132
- 2025Structural Foundations for Leading Digit Laws: Beyond Probabilistic Mixturescited by 2
- 2025Random Text, Zipf's Law, Critical Length,and Implications for Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.