# Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

arXiv: https://arxiv.org/abs/2602.14653 · HTML: https://mlsanity.com/paper/2602.14653/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset](https://mlsanity.com/paper/2205.12522/) (2022)
- [Revisiting the Uniform Information Density Hypothesis](https://mlsanity.com/paper/2109.11635/) (2021)
- [Morphology Matters: A Multilingual Language Modeling Analysis](https://mlsanity.com/paper/2012.06262/) (2020)
- [How to Compute the Probability of a Word](https://mlsanity.com/paper/2406.14561/) (2024)
- [Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks](https://mlsanity.com/paper/2210.14712/) (2022)
- [Generalized Measures of Anticipation and Responsivity in Online Language Processing](https://mlsanity.com/paper/2409.10728/) (2024)
- [Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives](https://mlsanity.com/paper/2310.13676/) (2023)
- [BNLP: Natural language processing toolkit for Bengali language](https://mlsanity.com/paper/2102.00405/) (2021)
- [A Cross-Linguistic Pressure for Uniform Information Density in Word Order](https://mlsanity.com/paper/2306.03734/) (2023)
- [Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse](https://mlsanity.com/paper/2410.16062/) (2024)
- [Towards a Similarity-adjusted Surprisal Theory](https://mlsanity.com/paper/2410.17676/) (2024)
- [The Harmonic Structure of Information Contours](https://mlsanity.com/paper/2506.03902/) (2025)
- [Revisiting Entropy Rate Constancy in Text](https://mlsanity.com/paper/2305.12084/) (2023)
- [An information-theoretic model of shallow and deep language comprehension](https://mlsanity.com/paper/2405.08223/) (2024)
- [The Impact of Token Granularity on the Predictive Power of Language Model Surprisal](https://mlsanity.com/paper/2412.11940/) (2024)
- [A Grounded Typology of Word Classes](https://mlsanity.com/paper/2412.10369/) (2024)

## Built on by (most cited first, 3 of 3)

- [Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in Dialogue](https://mlsanity.com/paper/2605.00506/) (2026)
- [Expect the Unexpected? Testing the Surprisal of Salient Entities](https://mlsanity.com/paper/2604.10724/) (2026)
- [The Dynamics of Human and AI-Generated Language: How Semantics Fluctuates across Different Timescales](https://mlsanity.com/paper/2606.11371/) (2026)

## Abstract (arXiv)

The Uniform Information Density (UID) hypothesis posits that speakers are subject to a communicative pressure to distribute information evenly within utterances, minimising surprisal variance. While this hypothesis has been tested empirically, prior studies are limited exclusively to text-only inputs, abstracting away from the perceptual context in which utterances are produced. In this work, we present the first computational study of UID in visually grounded settings. We estimate surprisal using multilingual vision-and-language models over image-caption data in 30 languages and visual storytelling data in 13 languages, together spanning 11 families. We find that grounding on perception consistently smooths the distribution of information, increasing both global and local uniformity across typologically diverse languages compared to text-only settings. In visual narratives, grounding in both image and discourse contexts has additional effects, with the strongest surprisal reductions occurring at the onset of discourse units. Overall, this study takes a first step towards modelling the temporal dynamics of information flow in ecologically plausible, multimodal language use, and finds that grounded language exhibits greater information uniformity, supporting a context-sensitive formulation of UID.