# Phonotactic Complexity and its Trade-offs

arXiv: https://arxiv.org/abs/2005.03774 · HTML: https://mlsanity.com/paper/2005.03774/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 23 of 23)

- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [A surprisal--duration trade-off across and within the world's languages](https://mlsanity.com/paper/2109.15000/) (2021)
- [An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech](https://mlsanity.com/paper/2306.02405/) (2023)
- [POWSM: A Phonetic Open Whisper-Style Speech Foundation Model](https://mlsanity.com/paper/2510.24992/) (2025)
- [How (Non-)Optimal is the Lexicon?](https://mlsanity.com/paper/2104.14279/) (2021)
- [Disambiguatory Signals are Stronger in Word-initial Positions](https://mlsanity.com/paper/2102.02183/) (2021)
- [Modeling the Unigram Distribution](https://mlsanity.com/paper/2106.02289/) (2021)
- [IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modeling](https://mlsanity.com/paper/2504.03036/) (2025)
- [Information-Theoretic Characterization of Vowel Harmony: A Cross-Linguistic Study on Word Lists](https://mlsanity.com/paper/2308.04885/) (2023)
- [PRiSM: Benchmarking Phone Realization in Speech Models](https://mlsanity.com/paper/2601.14046/) (2026)
- [Finding Concept-specific Biases in Form--Meaning Associations](https://mlsanity.com/paper/2104.06325/) (2021)
- [Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent](https://mlsanity.com/paper/2505.07659/) (2025)
- [On Homophony and R'enyi Entropy](https://mlsanity.com/paper/2109.13766/) (2021)
- [Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectiveness](https://mlsanity.com/paper/2306.02646/) (2023)
- [BabyLM's First Words: Word Segmentation as a Phonological Probing Task](https://mlsanity.com/paper/2504.03338/) (2025)
- [Minimal Effective Theory for Phonotactic Memory: Capturing Local Correlations due to Errors in Speech](https://mlsanity.com/paper/2309.02466/) (2023)
- [Transformer-Enabled Diachronic Analysis of Vedic Sanskrit: Neural Methods for Quantifying Types of Language Change](https://mlsanity.com/paper/2512.05364/) (2025)
- [The Distribution of Phoneme Frequencies across the World's Languages: Macroscopic and Microscopic Information-Theoretic Models](https://mlsanity.com/paper/2603.02860/) (2026)
- [Dependency distance minimization predicts compression](https://mlsanity.com/paper/2109.08900/) (2021)
- [NU HLT at CMCL 2022 Shared Task: Multilingual and Crosslingual Prediction of Human Reading Behavior in Universal Language Space](https://mlsanity.com/paper/2202.10855/) (2022)
- [Phonotactic Complexity across Dialects](https://mlsanity.com/paper/2402.12998/) (2024)
- [Modelling the Diachronic Emergence of Phoneme Frequency Distributions](https://mlsanity.com/paper/2603.09503/) (2026)
- [A Modular Architecture for Typologically Controlled Lexicon Generation](https://mlsanity.com/paper/2605.28824/) (2026)

## Abstract (arXiv)

We present methods for calculating a measure of phonotactic complexity---bits per phoneme---that permits a straightforward cross-linguistic comparison. When given a word, represented as a sequence of phonemic segments such as symbols in the international phonetic alphabet, and a statistical model trained on a sample of word types from the language, we can approximately measure bits per phoneme using the negative log-probability of that word under the model. This simple measure allows us to compare the entropy across languages, giving insight into how complex a language's phonotactics are. Using a collection of 1016 basic concept words across 106 languages, we demonstrate a very strong negative correlation of -0.74 between bits per phoneme and the average length of words.