# A Statistical Model of Word Rank Evolution

arXiv: https://arxiv.org/abs/2107.09948 · HTML: https://mlsanity.com/paper/2107.09948/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 76 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 10 of 10)

- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics](https://mlsanity.com/paper/1812.08092/) (2018)
- [Temporal Analysis of Language through Neural Language Models](https://mlsanity.com/paper/1405.3515/) (2014)
- [Dynamic Word Embeddings](https://mlsanity.com/paper/1702.08359/) (2017)
- [Human language reveals a universal positivity bias](https://mlsanity.com/paper/1406.3855/) (2014)
- [Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution](https://mlsanity.com/paper/1501.00960/) (2015)
- [Evolutionary forces in language change](https://mlsanity.com/paper/1608.00938/) (2016)
- [The Natural Selection of Words: Finding the Features of Fitness](https://mlsanity.com/paper/1908.07013/) (2019)
- [Rank diversity of languages: Generic behavior in computational linguistics](https://mlsanity.com/paper/1505.03783/) (2015)

## Abstract (arXiv)

The availability of large linguistic data sets enables data-driven approaches to study linguistic change. The Google Books corpus unigram frequency data set is used to investigate the word rank dynamics in eight languages. We observed the rank changes of the unigrams from 1900 to 2008 and compared it to a Wright-Fisher inspired model that we developed for our analysis. The model simulates a neutral evolutionary process with the restriction of having no disappearing and added words. This work explains the mathematical framework of the model - written as a Markov Chain with multinomial transition probabilities - to show how frequencies of words change in time. From our observations in the data and our model, word rank stability shows two types of characteristics: (1) the increase/decrease in ranks are monotonic, or (2) the rank stays the same. Based on our model, high-ranked words tend to be more stable while low-ranked words tend to be more volatile. Some words change in ranks in two ways: (a) by an accumulation of small increasing/decreasing rank changes in time and (b) by shocks of increase/decrease in ranks. Most words in all of the languages we have looked at are rank stable, but not as stable as a neutral model would predict. The stopwords and Swadesh words are observed to be rank stable across eight languages indicating linguistic conformity in established languages. These signatures suggest unigram frequencies in all languages have changed in a manner inconsistent with a purely neutral evolutionary process.