# Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems

arXiv: https://arxiv.org/abs/2205.01128 · HTML: https://mlsanity.com/paper/2205.01128/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 72 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019)
- [Linguistic generalization and compositionality in modern artificial neural networks](https://mlsanity.com/paper/1904.00157/) (2019)
- [Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving](https://mlsanity.com/paper/1910.06611/) (2019)
- [Discovering the Compositional Structure of Vector Representations with Role Learning Networks](https://mlsanity.com/paper/1910.09113/) (2019)
- [Learning to Reason with Third-Order Tensor Products](https://mlsanity.com/paper/1811.12143/) (2018)
- [RNNs Implicitly Implement Tensor Product Representations](https://mlsanity.com/paper/1812.08718/) (2018)
- [The Unstoppable Rise of Computational Linguistics in Deep Learning](https://mlsanity.com/paper/2005.06420/) (2020)
- [Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization](https://mlsanity.com/paper/2106.01317/) (2021)
- [Question-Answering with Grammatically-Interpretable Representations](https://mlsanity.com/paper/1705.08432/) (2017)
- [Attentive Tensor Product Learning](https://mlsanity.com/paper/1802.07089/) (2018)
- [Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations](https://mlsanity.com/paper/1811.01062/) (2018)
- [A Century Long Commitment to Assessing Artificial Intelligence and its Impact on Society](https://mlsanity.com/paper/1808.07899/) (2018)

## Built on by (most cited first, 30 of 44)

- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Does CLIP Bind Concepts? Probing Compositionality in Large Image Models](https://mlsanity.com/paper/2212.10537/) (2022)
- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Break It Down: Evidence for Structural Compositionality in Neural Networks](https://mlsanity.com/paper/2301.10884/) (2023)
- [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [Neural-Logic Human-Object Interaction Detection](https://mlsanity.com/paper/2311.09817/) (2023)
- [A polar coordinate system represents syntax in large language models](https://mlsanity.com/paper/2412.05571/) (2024)
- [A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates](https://mlsanity.com/paper/2401.03910/) (2024)
- [Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers](https://mlsanity.com/paper/2501.08537/) (2025)
- [Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks](https://mlsanity.com/paper/2210.00400/) (2022)
- [Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures](https://mlsanity.com/paper/2205.09208/) (2022)
- [Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing](https://mlsanity.com/paper/2405.05409/) (2024)
- [A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practices](https://mlsanity.com/paper/2406.08787/) (2024)
- [On The Specialization of Neural Modules](https://mlsanity.com/paper/2409.14981/) (2024)
- [Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks](https://mlsanity.com/paper/2406.01895/) (2024)
- [Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models](https://mlsanity.com/paper/2210.03575/) (2022)
- [Probabilistic Abduction for Visual Abstract Reasoning via Learning Rules in Vector-symbolic Architectures](https://mlsanity.com/paper/2401.16024/) (2024)
- [From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks](https://mlsanity.com/paper/2405.15164/) (2024)
- [Revealing emergent human-like conceptual representations from language prediction](https://mlsanity.com/paper/2501.12547/) (2025)
- [A Theory of Initialisation's Impact on Specialisation](https://mlsanity.com/paper/2503.02526/) (2025)
- [Recursive Neural Programs: Variational Learning of Image Grammars and Part-Whole Hierarchies](https://mlsanity.com/paper/2206.08462/) (2022)
- [The Hyperdimensional Transform: a Holographic Representation of Functions](https://mlsanity.com/paper/2310.16065/) (2023)
- [Hyperdimensional computing: a fast, robust and interpretable paradigm for biological data](https://mlsanity.com/paper/2402.17572/) (2024)
- [Hey Pentti, We Did It!: A Fully Vector-Symbolic Lisp](https://mlsanity.com/paper/2510.17889/) (2025)
- [HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Models](https://mlsanity.com/paper/2305.09948/) (2023)
- [Semantic Composition in Visually Grounded Language Models](https://mlsanity.com/paper/2305.16328/) (2023)
- [Intelligence-Endogenous Management Platform for Computing and Network Convergence](https://mlsanity.com/paper/2308.03450/) (2023)
- [Terminating Differentiable Tree Experts](https://mlsanity.com/paper/2407.02060/) (2024)
- [Explainability Through Systematicity: The Hard Systematicity Challenge for Artificial Intelligence](https://mlsanity.com/paper/2507.22197/) (2025)

## Abstract (arXiv)

What explains the dramatic progress from 20th-century to 21st-century AI, and how can the remaining limitations of current AI be overcome? The widely accepted narrative attributes this progress to massive increases in the quantity of computational and data resources available to support statistical learning in deep artificial neural networks. We show that an additional crucial factor is the development of a new type of computation. Neurocompositional computing adopts two principles that must be simultaneously respected to enable human-level cognition: the principles of Compositionality and Continuity. These have seemed irreconcilable until the recent mathematical discovery that compositionality can be realized not only through discrete methods of symbolic computing, but also through novel forms of continuous neural computing. The revolutionary recent progress in AI has resulted from the use of limited forms of neurocompositional computing. New, deeper forms of neurocompositional computing create AI systems that are more robust, accurate, and comprehensible.