On the Emergence of Syntax by Means of Local Interaction
Not on the map. Why?
Abstract (arXiv)
Can syntactic processing emerge spontaneously from purely local interaction? We present a concrete instance on a minimal system: an 18,658-parameter two-dimensional neural cellular automaton (NCA), supervised by nothing more than a 1-bit boundary signal, is trained on the membership problem of an arithmetic-expression grammar. After training, its internal L × L grid spontaneously self-organizes into an ordered, spatially extended representation that we name Proto-CKY. This representation satisfies three operational criteria for syntactic processing: expressive power beyond the regular languages, structural generalization beyond the training distribution, and an internal organization quantitatively aligned with grammatical structure (Pearson r ≈ 0.71). It emerges independently on four context-free grammars and regenerates spontaneously after perturbation. Proto-CKY is functionally aligned with the CKY algorithm but formally distinct from it: it is a physical prototype, a concrete instantiation of a mathematical ideal on a physical substrate, and the systematic distance between the two carries information about the substrate itself.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
23 of 42 references.
References · 23
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
- 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
- 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
- 2018Targeted Syntactic Evaluation of Language Modelscited by 239
- 2022Neural Networks and the Chomsky Hierarchycited by 214
11 more
- 2020Information-Theoretic Probing for Linguistic Structurecited by 204
- 2022Exploring Length Generalization in Large Language Modelscited by 204
- 2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
- 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
- 2017Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamplescited by 66
- 2020RNNs can generate bounded hierarchical languages with optimal memorycited by 59
- 2021Growing 3D Artefacts and Functional Machines with Neural Cellular Automatacited by 38
- 2019Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languagescited by 31
- 2022Attention-based Neural Cellular Automatacited by 26
- 2021Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agentcited by 15
- 2026On the Spatiotemporal Dynamics of Generalization in Neural Networkscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Structural Generalization on SLOG without Hand-Written Rules28 Apr 2026