# Natural Language Syntax Complies with the Free-Energy Principle

arXiv: https://arxiv.org/abs/2210.15098 · HTML: https://mlsanity.com/paper/2210.15098/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 292 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 7 of 7)

- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [A World unto Itself: Human Communication as Active Inference](https://mlsanity.com/paper/1906.10538/) (2019)
- [How Robust are Deep Neural Networks?](https://mlsanity.com/paper/1804.11313/) (2018)
- [Topological Analysis of Syntactic Structures](https://mlsanity.com/paper/1903.05181/) (2019)
- [Language Design as Information Renormalization](https://mlsanity.com/paper/1708.01525/) (2017)
- [Neural dynamics under active inference: plausibility and efficiency of information processing](https://mlsanity.com/paper/2001.08028/) (2020)
- [LT^2C^2: A language of thought with Turing-computable Kolmogorov complexity](https://mlsanity.com/paper/1303.0875/) (2013)

## Built on by (most cited first, 7 of 7)

- [A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?](https://mlsanity.com/paper/2308.00109/) (2023)
- [Syntactic Language Change in English and German: Metrics, Parsers, and Convergences](https://mlsanity.com/paper/2402.11549/) (2024)
- [A Comparative Investigation of Compositional Syntax and Semantics in DALL-E 2](https://mlsanity.com/paper/2403.12294/) (2024)
- [Neurocomputational Mechanisms of Syntactic Transfer in Bilingual Sentence Production](https://mlsanity.com/paper/2601.18056/) (2026)
- [Merge-based syntax is mediated by distinct neurocognitive mechanisms: A clustering analysis of comprehension abilities in 84,000 individuals with language deficits across nine languages](https://mlsanity.com/paper/2508.02885/) (2025)
- [Graph Distance as Surprise: Free Energy Minimization in Knowledge Graph Reasoning](https://mlsanity.com/paper/2512.01878/) (2025)
- [A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models](https://mlsanity.com/paper/2605.09515/) (2026)

## Abstract (arXiv)

Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design - such as "minimal search" criteria from theoretical syntax - adhere to the FEP. This affords a greater degree of explanatory power to the FEP - with respect to higher language functions - and offers linguistics a grounding in first principles with respect to computability. We show how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel-Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing-Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference.