On the Computational Complexity of Structural Generalization
Not on the map. Why?
Abstract (arXiv)
Structural generalization has been measured repeatedly by several benchmarks, yet it has never been formally defined. We give a definition that translates the two premises (compositional structure and unbounded generalization) into mathematical language. The definition itself is neutral: a compiler that hard-codes the rules satisfies it just as well. But structural generalization becomes a scientific question only insofar as the capacity can autonomously emerge from finite data. This question pits the computational lower bound NC1 against the learnable ceiling TC0 of pure Transformers. Under a Montagovian instantiation, each compositional rule splits into two projections: a syntactic face (Fγ) and a semantic face (Gγ). Tree evaluation on the Gγ side is an instantiation of BFVP, which is NC1-complete (Buss, 1987). A pure Transformer must learn both faces at once, but Kraus et al. (2026) prove that its learnable class subseteq TC0. Under the standard assumption TC0 neq NC1, a pure Transformer cannot learn structural generalization. Neuro-symbolic systems achieve the best benchmark scores precisely because they inject Gγ, sidestepping the genuinely hard half. Benchmark scores cannot distinguish "learned" from "given." This is what this paper sets out to make clear.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 21 Jul 2026.
Builds on
11 of 24 references.
References · 11
- 2019Compositionality decomposed: how do neural networks generalise?cited by 264
- 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
- 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
- 2019On the Turing Completeness of Modern Neural Network Architecturescited by 129
- 2024Why are Sensitive Functions Hard for Transformers?cited by 74
- 2022Structural generalization is hard for sequence-to-sequence modelscited by 24
- 2023SLOG: A Structural Generalization Benchmark for Semantic Parsingcited by 22
- 2018AMR Dependency Parsing with a Typed Semantic Algebracited by 17
- 2025Compositional Reasoning with Transformers, RNNs, and Chain of Thoughtcited by 6
- 2024Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformationscited by 4
- 2026Barriers to Universal Reasoning With Transformers (And How to Overcome Them)cited by 1
Built on by
Not cited yet.