# Identifying the Periodicity of Information in Natural Language

arXiv: https://arxiv.org/abs/2510.27241 · HTML: https://mlsanity.com/paper/2510.27241/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 42 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 8 of 8)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Revisiting the Uniform Information Density Hypothesis](https://mlsanity.com/paper/2109.11635/) (2021)
- [Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihood](https://mlsanity.com/paper/2406.19874/) (2024)
- [Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse](https://mlsanity.com/paper/2410.16062/) (2024)
- [GCDT: A Chinese RST Treebank for Multigenre and Multilingual Discourse Parsing](https://mlsanity.com/paper/2210.10449/) (2022)
- [Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition](https://mlsanity.com/paper/2012.12007/) (2020)
- [The Harmonic Structure of Information Contours](https://mlsanity.com/paper/2506.03902/) (2025)
- [FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy](https://mlsanity.com/paper/2305.10307/) (2023)

## Built on by (most cited first, 4 of 4)

- [LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series](https://mlsanity.com/paper/2605.20449/) (2026)
- [The Dynamics of Human and AI-Generated Language: How Semantics Fluctuates across Different Timescales](https://mlsanity.com/paper/2606.11371/) (2026)
- [Modeling the Arrival Time of Information-Bearing Events in Natural Language](https://mlsanity.com/paper/2610.14212/) (2026)
- [Uniform Information Density-Based Preference Optimization](https://mlsanity.com/paper/2610.14578/) (2026)

## Abstract (arXiv)

Recent theoretical advancement of information density in natural language has brought the following question on desk: To what degree does natural language exhibit periodicity pattern in its encoded information? We address this question by introducing a new method called AutoPeriod of Surprisal (APS). APS adopts a canonical periodicity detection algorithm and is able to identify any significant periods that exist in the surprisal sequence of a single document. By applying the algorithm to a set of corpora, we have obtained the following interesting results: Firstly, a considerable proportion of human language demonstrates a strong pattern of periodicity in information; Secondly, new periods that are outside the distributions of typical structural units in text (e.g., sentence boundaries, elementary discourse units, etc.) are found and further confirmed via harmonic regression modeling. We conclude that the periodicity of information in language is a joint outcome from both structured factors and other driving factors that take effect at longer distances. The advantages of our periodicity detection method and its potentials in LLM-generation detection are further discussed.