# Generalized Entropies and the Similarity of Texts

arXiv: https://arxiv.org/abs/1611.03596 · HTML: https://mlsanity.com/paper/1611.03596/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2016.

## Built on by (most cited first, 3 of 3)

- [Entropy and type-token ratio in gigaword corpora](https://mlsanity.com/paper/2411.10227/) (2024)
- [Multilayer Networks for Text Analysis with Multiple Data Types](https://mlsanity.com/paper/2106.15821/) (2021)
- [Quantifying the Dissimilarity of Texts](https://mlsanity.com/paper/2305.02457/) (2023)

## Abstract (arXiv)

We show how generalized Gibbs-Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf's law) implies that the generalized entropies, computed at the word level, are dominated by words in a specific range of frequencies. Here we show that this is the case not only for the generalized entropies but also for the generalized (Jensen-Shannon) divergences, used to compute the similarity between different texts. This finding allows us to identify the contribution of specific words (and word frequencies) for the different generalized entropies and also to estimate the size of the databases needed to obtain a reliable estimation of the divergences. We test our results in large databases of books (from the Google n-gram database) and scientific papers (indexed by Web of Science).