# Corrections of Zipf's and Heaps' Laws Derived from Hapax Rate Models

arXiv: https://arxiv.org/abs/2307.12896 · HTML: https://mlsanity.com/paper/2307.12896/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 46 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy](https://mlsanity.com/paper/2302.09049/) (2023)
- [Text mixing shapes the anatomy of rank-frequency distributions: A modern Zipfian mechanics for natural language](https://mlsanity.com/paper/1409.3870/) (2014)
- [Types, Tokens, and Hapaxes: A New Heap's Law](https://mlsanity.com/paper/1901.00521/) (2018)

## Built on by (most cited first, 3 of 3)

- [From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis](https://mlsanity.com/paper/2512.13491/) (2025)
- [Quadratic Term Correction on Heaps' Law](https://mlsanity.com/paper/2511.14683/) (2025)
- [How to measure the optimality of word or gesture order with respect to the principle of swap distance minimization](https://mlsanity.com/paper/2604.01938/) (2026)

## Abstract (arXiv)

The article introduces corrections to Zipf's and Heaps' laws based on systematic models of the proportion of hapaxes, i.e., words that occur once. The derivation rests on two assumptions. The first one is the standard urn model which predicts that marginal frequency distributions for shorter texts look as if word tokens were sampled blindly from a given longer text. The second assumption posits that the hapax rate is a simple function of the text length. Four such functions are discussed: the constant model, the cancelation model, the linear model, and the logistic model. As a simple illustration, it is shown that the logistic model yields the best fit for a sample of 14 texts in English. The need and the availability of more complex mixture models that reflect two-regime vocabularies for larger corpora is also discussed.