# Zipf's law holds for phrases, not words

arXiv: https://arxiv.org/abs/1406.5181 · HTML: https://mlsanity.com/paper/1406.5181/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2014.

## Built on by (most cited first, 12 of 12)

- [Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models](https://mlsanity.com/paper/2402.19449/) (2024)
- [UCPhrase: Unsupervised Context-aware Quality Phrase Tagging](https://mlsanity.com/paper/2105.14078/) (2021)
- [Complex systems approach to natural language](https://mlsanity.com/paper/2401.02772/) (2024)
- [Sentiment and structure in word co-occurrence networks on Twitter](https://mlsanity.com/paper/2110.00587/) (2021)
- [A decomposition of book structure through ousiometric fluctuations in cumulative word-time](https://mlsanity.com/paper/2208.09496/) (2022)
- [Zipf-Gramming: Scaling Byte N-Grams Up to Production Sized Malware Corpora](https://mlsanity.com/paper/2511.13808/) (2025)
- [Scaling Efficient LLMs](https://mlsanity.com/paper/2402.14746/) (2024)
- [Computational Paremiology: Charting the temporal, ecological dynamics of proverb use in books, news articles, and tweets](https://mlsanity.com/paper/2107.04929/) (2021)
- [Ousiometrics: The essence of meaning aligns with a power-danger-structure framework instead of valence-arousal-dominance](https://mlsanity.com/paper/2110.06847/) (2021)
- [Approaches to the classification of complex systems: Words, texts, and more](https://mlsanity.com/paper/2205.04060/) (2022)
- [Subdiffusive semantic evolution in Indo-European languages](https://mlsanity.com/paper/2209.04701/) (2022)
- [Big Data and Large Numbers. Interpreting Zipf's Law](https://mlsanity.com/paper/2305.02687/) (2023)

## Abstract (arXiv)

With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in scaling. Here, building on the simple observation that phrases of one or more words comprise the most coherent units of meaning in language, we show empirically that Zipf's law for phrases extends over as many as nine orders of rank magnitude. In doing so, we develop a principled and scalable statistical mechanical method of random text partitioning, which opens up a rich frontier of rigorous text analysis via a rank ordering of mixed length phrases.