mlsanityData to 14 Oct 2026

The Harmonic Structure of Information Contours

arXiv 2506.03902 ↗First posted 4 Jun 2025cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

The uniform information density (UID) hypothesis proposes that speakers aim to distribute information evenly throughout a text, balancing production effort and listener comprehension difficulty. However, language typically does not maintain a strictly uniform information rate; instead, it fluctuates around a global average. These fluctuations are often explained by factors such as syntactic constraints, stylistic choices, or audience design. In this work, we explore an alternative perspective: that these fluctuations may be influenced by an implicit linguistic pressure towards periodicity, where the information rate oscillates at regular intervals, potentially across multiple frequencies simultaneously. We apply harmonic regression and introduce a novel extension called time scaling to detect and test for such periodicity in information contours. Analyzing texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese, we find consistent evidence of periodic patterns in information rate. Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization. Beyond highlighting the connection between information rate and discourse structure, our approach offers a general framework for uncovering structural pressures at various levels of linguistic granularity.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 15 of its 83 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

15 of 83 references.

References · 15

  1. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 119
  2. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
  3. 2021Revisiting the Uniform Information Density Hypothesiscited by 68
  4. 2024Latxa: An Open Language Model and Evaluation Suite for Basquecited by 42
  5. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  6. 2024From Language Models over Tokens to Language Models over Characterscited by 31
  7. 2024Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihoodcited by 24
  8. 2024Generalized Measures of Anticipation and Responsivity in Online Language Processingcited by 24
  9. 2023Information Value: Measuring Utterance Predictability as Distance from Plausible Alternativescited by 23
  10. 2023Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentationcited by 17
  11. 2024Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discoursecited by 17
  12. 2024Towards a Similarity-adjusted Surprisal Theorycited by 9
3 more
  1. 2023Revisiting Entropy Rate Constancy in Textcited by 7
  2. 2024From "um" to "yeah": Producing, predicting, and regulating information flow in human conversationcited by 7
  3. 2024An information-theoretic model of shallow and deep language comprehensioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.