mlsanityData to 7 Oct 2026

Natural Language Syntax Complies with the Free-Energy Principle

arXiv 2210.15098 ↗First posted 27 Oct 2022cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design - such as "minimal search" criteria from theoretical syntax - adhere to the FEP. This affords a greater degree of explanatory power to the FEP - with respect to higher language functions - and offers linguistics a grounding in first principles with respect to computability. We show how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel-Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing-Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 7 of its 292 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

7 of 292 references.

References · 7

  1. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  2. 2019A World unto Itself: Human Communication as Active Inferencecited by 11
  3. 2018How Robust are Deep Neural Networks?cited by 10
  4. 2019Topological Analysis of Syntactic Structurescited by 9
  5. 2017Language Design as Information Renormalizationcited by 6
  6. 2020Neural dynamics under active inference: plausibility and efficiency of information processingcited by 5
  7. 2013LT^2C^2: A language of thought with Turing-computable Kolmogorov complexitycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 2.

7 papers.