# Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent

arXiv: https://arxiv.org/abs/2505.07659 · HTML: https://mlsanity.com/paper/2505.07659/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Testing the Predictions of Surprisal Theory in 11 Languages](https://mlsanity.com/paper/2307.03667/) (2023)
- [Phonotactic Complexity and its Trade-offs](https://mlsanity.com/paper/2005.03774/) (2020)
- [Meaning to Form: Measuring Systematicity as Information](https://mlsanity.com/paper/1906.05906/) (2019)
- [Quantifying the redundancy between prosody and text](https://mlsanity.com/paper/2311.17233/) (2023)
- [Multilingual Gradient Word-Order Typology from Universal Dependencies](https://mlsanity.com/paper/2402.01513/) (2024)
- [Revisiting the Optimality of Word Lengths](https://mlsanity.com/paper/2312.03897/) (2023)
- [Information-Theoretic Characterization of Vowel Harmony: A Cross-Linguistic Study on Word Lists](https://mlsanity.com/paper/2308.04885/) (2023)
- [The time scale of redundancy between prosody and linguistic context](https://mlsanity.com/paper/2503.11630/) (2025)
- [Predicting cross-linguistic adjective order with information gain](https://mlsanity.com/paper/2012.15263/) (2020)
- [A Comparative Study of Transformers on Word Sense Disambiguation](https://mlsanity.com/paper/2111.15417/) (2021)

## Built on by (most cited first, 3 of 3)

- [What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels](https://mlsanity.com/paper/2512.16832/) (2025)
- [Modeling Bottom-up Information Quality during Language Processing](https://mlsanity.com/paper/2509.17047/) (2025)
- [Using Prosody to Predict Syntactic Structure](https://mlsanity.com/paper/2608.30260/) (2026)

## Abstract (arXiv)

This paper argues that the relationship between lexical identity and prosody -- one well-studied parameter of linguistic variation -- can be characterized using information theory. We predict that languages that use prosody to make lexical distinctions should exhibit a higher mutual information between word identity and prosody, compared to languages that don't. We test this hypothesis in the domain of pitch, which is used to make lexical distinctions in tonal languages, like Cantonese. We use a dataset of speakers reading sentences aloud in ten languages across five language families to estimate the mutual information between the text and their pitch curves. We find that, across languages, pitch curves display similar amounts of entropy. However, these curves are easier to predict given their associated text in the tonal languages, compared to pitch- and stress-accent languages, and thus the mutual information is higher in these languages, supporting our hypothesis. Our results support perspectives that view linguistic typology as gradient, rather than categorical.