# Cognition is All You Need -- The Next Layer of AI Above Large Language Models

arXiv: https://arxiv.org/abs/2403.02164 · HTML: https://mlsanity.com/paper/2403.02164/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2024.

## Built on by (most cited first, 3 of 3)

- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [A Scoping Review of Large Language Model-Based Pedagogical Agents](https://mlsanity.com/paper/2604.12253/) (2026)
- [SymSeqBench: a unified framework for the generation and analysis of rule-based symbolic sequences and datasets](https://mlsanity.com/paper/2512.24977/) (2025)

## Abstract (arXiv)

Recent studies of the applications of conversational AI tools, such as chatbots powered by large language models, to complex real-world knowledge work have shown limitations related to reasoning and multi-step problem solving. Specifically, while existing chatbots simulate shallow reasoning and understanding they are prone to errors as problem complexity increases. The failure of these systems to address complex knowledge work is due to the fact that they do not perform any actual cognition. In this position paper, we present Cognitive AI, a higher-level framework for implementing programmatically defined neuro-symbolic cognition above and outside of large language models. Specifically, we propose a dual-layer functional architecture for Cognitive AI that serves as a roadmap for AI systems that can perform complex multi-step knowledge work. We propose that Cognitive AI is a necessary precursor for the evolution of higher forms of AI, such as AGI, and specifically claim that AGI cannot be achieved by probabilistic approaches on their own. We conclude with a discussion of the implications for large language models, adoption cycles in AI, and commercial Cognitive AI development.