mlsanityData to 7 Oct 2026

On the Emergence of Syntax by Means of Local Interaction

arXiv 2604.17857 ↗First posted 20 Apr 2026cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Can syntactic processing emerge spontaneously from purely local interaction? We present a concrete instance on a minimal system: an 18,658-parameter two-dimensional neural cellular automaton (NCA), supervised by nothing more than a 1-bit boundary signal, is trained on the membership problem of an arithmetic-expression grammar. After training, its internal L × L grid spontaneously self-organizes into an ordered, spatially extended representation that we name Proto-CKY. This representation satisfies three operational criteria for syntactic processing: expressive power beyond the regular languages, structural generalization beyond the training distribution, and an internal organization quantitatively aligned with grammatical structure (Pearson r ≈ 0.71). It emerges independently on four context-free grammars and regenerates spontaneously after perturbation. Proto-CKY is functionally aligned with the CKY algorithm but formally distinct from it: it is a physical prototype, a concrete instantiation of a mathematical ideal on a physical substrate, and the systematic distance between the two carries information about the substrate itself.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

23 of 42 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  6. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  7. 2019Designing and Interpreting Probes with Control Taskscited by 608
  8. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  9. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  10. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  11. 2018Targeted Syntactic Evaluation of Language Modelscited by 239
  12. 2022Neural Networks and the Chomsky Hierarchycited by 214
11 more
  1. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  2. 2022Exploring Length Generalization in Large Language Modelscited by 204
  3. 2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
  4. 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
  5. 2017Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamplescited by 66
  6. 2020RNNs can generate bounded hierarchical languages with optimal memorycited by 59
  7. 2021Growing 3D Artefacts and Functional Machines with Neural Cellular Automatacited by 38
  8. 2019Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languagescited by 31
  9. 2022Attention-based Neural Cellular Automatacited by 26
  10. 2021Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agentcited by 15
  11. 2026On the Spatiotemporal Dynamics of Generalization in Neural Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.