mlsanityData to 7 Oct 2026

Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems

arXiv 2205.01128 ↗First posted 2 May 2022cs.AI cs.NE cs.SCcited by 44

Not on the map. Why?

Abstract (arXiv)

What explains the dramatic progress from 20th-century to 21st-century AI, and how can the remaining limitations of current AI be overcome? The widely accepted narrative attributes this progress to massive increases in the quantity of computational and data resources available to support statistical learning in deep artificial neural networks. We show that an additional crucial factor is the development of a new type of computation. Neurocompositional computing adopts two principles that must be simultaneously respected to enable human-level cognition: the principles of Compositionality and Continuity. These have seemed irreconcilable until the recent mathematical discovery that compositionality can be realized not only through discrete methods of symbolic computing, but also through novel forms of continuous neural computing. The revolutionary recent progress in AI has resulted from the use of limited forms of neurocompositional computing. New, deeper forms of neurocompositional computing create AI systems that are more robust, accurate, and comprehensible.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 18 of its 72 references that are in the map. First posted on arXiv 2 May 2022.

Builds on

18 of 72 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  3. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  4. 2015Neural Module Networkscited by 375
  5. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  6. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
  7. 2019Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoningcited by 149
  8. 2019Linguistic generalization and compositionality in modern artificial neural networkscited by 58
  9. 2019Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solvingcited by 34
  10. 2019Discovering the Compositional Structure of Vector Representations with Role Learning Networkscited by 33
  11. 2018Learning to Reason with Third-Order Tensor Productscited by 27
  12. 2018RNNs Implicitly Implement Tensor Product Representationscited by 25
6 more
  1. 2020The Unstoppable Rise of Computational Linguistics in Deep Learningcited by 12
  2. 2021Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarizationcited by 11
  3. 2017Question-Answering with Grammatically-Interpretable Representationscited by 8
  4. 2018Attentive Tensor Product Learningcited by 6
  5. 2018Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representationscited by 2
  6. 2018A Century Long Commitment to Assessing Artificial Intelligence and its Impact on Societycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 7; 2024: 10; 2025: 11; 2026: 9.

44 papers.

Sort:

By area:

  1. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  2. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  3. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
  4. 2023Break It Down: Evidence for Structural Compositionality in Neural Networkscited by 44
  5. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 40
  6. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 34
  7. 2023Neural-Logic Human-Object Interaction Detectioncited by 25
  8. 2024A polar coordinate system represents syntax in large language modelscited by 24
  9. 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debatescited by 22
  10. 2022Systematic Generalization and Emergent Structures in Transformers Trained on Structured Taskscited by 19
  11. 2025Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformerscited by 19
  12. 2022Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architecturescited by 17
32 more
  1. 2024Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizingcited by 15
  2. 2024A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practicescited by 15
  3. 2024On The Specialization of Neural Modulescited by 14
  4. 2024Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Taskscited by 10
  5. 2022Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Modelscited by 9
  6. 2024Probabilistic Abduction for Visual Abstract Reasoning via Learning Rules in Vector-symbolic Architecturescited by 9
  7. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  8. 2025A Theory of Initialisation's Impact on Specialisationcited by 6
  9. 2025Revealing emergent human-like conceptual representations from language predictioncited by 5
  10. 2022Recursive Neural Programs: Variational Learning of Image Grammars and Part-Whole Hierarchiescited by 2
  11. 2023The Hyperdimensional Transform: a Holographic Representation of Functionscited by 2
  12. 2024Hyperdimensional computing: a fast, robust and interpretable paradigm for biological datacited by 2
  13. 2025Hey Pentti, We Did It!: A Fully Vector-Symbolic Lispcited by 2
  14. 2023HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Modelscited by 1
  15. 2023Semantic Composition in Visually Grounded Language Modelscited by 1
  16. 2023Intelligence-Endogenous Management Platform for Computing and Network Convergencecited by 1
  17. 2024Terminating Differentiable Tree Expertscited by 1
  18. 2025Explainability Through Systematicity: The Hard Systematicity Challenge for Artificial Intelligencecited by 0
  19. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
  20. 2025Large language models have learned to use languagecited by 0
  21. 2025Large language models are not about natural languagecited by 0
  22. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  23. 2025Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AIcited by 0
  24. 2026Are Language Models Models?cited by 0
  25. 2026Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMscited by 0
  26. 2026A logical re-conception of neural networks: Hamiltonian bitwise part-whole architecturecited by 0
  27. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  28. 2026You Can't Fight in Here! This is BBS!cited by 0
  29. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0
  30. 2026Tracing Computation Density in LLMscited by 0
  31. 2026The New Associationism: Lessons from Deep Learningcited by 0
  32. 2026TPR-Attention for Combinatorial Generalizationcited by 0