mlsanityData to 7 Oct 2026

Cognitive Foundations for Reasoning and Their Manifestation in LLMs

arXiv 2511.16660 ↗First posted 20 Nov 2025cs.AIcited by 26

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning. To understand this gap, we synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations. We introduce a fine-grained evaluation framework and conduct the first large-scale empirical analysis of 192K traces from 18 models across text, vision, and audio, complemented by 54 human think-aloud traces, which we make publicly available. We find that models under-utilize cognitive elements correlated with success, narrowing to rigid sequential processing on ill-structured problems where diverse representations and meta-cognitive monitoring are critical. Human traces show more abstraction and conceptual processing, while models default to surface-level enumeration. Meta-analysis of 1.6K LLM reasoning papers reveals the research community concentrates on easily quantifiable elements (sequential organization: 55%, decomposition: 60%) but neglecting meta-cognitive controls (self-awareness: 16%) that correlate with success. Models possess behavioral repertoires associated with success but fail to deploy them spontaneously. Leveraging these patterns, we develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems. By establishing a shared vocabulary between cognitive science and LLM research, our framework enables systematic diagnosis of reasoning failures and principled development of models that reason through robust cognitive mechanisms rather than spurious shortcuts, while providing tools to test theories of human cognition at scale.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 43 of its 181 references that are in the map. First posted on arXiv 20 Nov 2025.

Builds on

43 of 181 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025OpenThoughts: Data Recipes for Reasoning Models4 Jun
  2. 2025Qwen3 Technical Report14 May
  3. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning2 Apr

Other references · 40

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  9. 2025s1: Simple test-time scalingcited by 1,311
  10. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  11. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  12. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
28 more
  1. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  2. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  4. 2024A Survey on Large Language Models for Code Generationcited by 646
  5. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  6. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  7. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  8. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  9. 2024The pitfalls of next-token predictioncited by 156
  10. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 113
  11. 2025Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2cited by 95
  12. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 81
  13. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 78
  14. 2025MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligencecited by 59
  15. 2022Bidirectional Language Models Are Also Few-shot Learnerscited by 48
  16. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  17. 2025Fluid Language Model Benchmarkingcited by 30
  18. 2025Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Trainingcited by 29
  19. 2025To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoningcited by 24
  20. 2025PrefPalette: Personalized Preference Modeling with Latent Attributescited by 15
  21. 2025Potemkin Understanding in Large Language Modelscited by 14
  22. 2025BLAB: Brutally Long Audio Benchcited by 11
  23. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  24. 2025PrefDisco: Benchmarking Proactive Personalized Reasoningcited by 7
  25. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Modelscited by 6
  26. 2016The Language of Generalizationcited by 2
  27. 2025Can Memory-Augmented Language Models Generalize on Reasoning-in-a-Haystack Tasks?cited by 2
  28. 2025On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Studycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 24.

26 papers.

Sort:

By area:

  1. 2026Vero: An Open RL Recipe for General Visual Reasoningcited by 10
  2. 2025Schoenfeld's Anatomy of Mathematical Reasoning by Language Modelscited by 8
  3. 2026Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoningcited by 7
  4. 2026ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Tracescited by 5
  5. 2026Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspirationcited by 4
  6. 2026ReasonOps: Operator Segmentation for LLM Reasoning Tracescited by 4
  7. 2026Learning User Simulators with Turing Rewardscited by 3
  8. 2026Chain of Mindset: Reasoning with Adaptive Cognitive Modescited by 2
  9. 2026Cognitive Models and AI Algorithms Provide Templates for Designing Language Agentscited by 2
  10. 2026Deep Reasoning in General Purpose Agents via Structured Meta-Cognitioncited by 2
  11. 2026UniCog: Uncovering Cognitive Abilities of LLMs through Latent Mind Space Analysiscited by 1
  12. 2026Faithful Mobile GUI Agents with Guided Advantage Estimatorcited by 1
14 more
  1. 2026Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMscited by 1
  2. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  3. 2025Teaching and Critiquing Conceptualization and Operationalization in NLPcited by 0
  4. 2026Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answeringcited by 0
  5. 2026Improving reasoning at inference time via uncertainty minimisationcited by 0
  6. 2026ThinkProbe: Beyond Accuracy -- Structural Profiling of Open-Ended LLM Reasoning Traces via Non-Generative Thought Graphscited by 0
  7. 2026Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learningcited by 0
  8. 2026Mixture of Cognitive Experts in Large Vision-Language Modelscited by 0
  9. 2026Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcementcited by 0
  10. 2026Cognitive Demand Steering for Adaptive Meta-Reasoning in Large Language Modelscited by 0
  11. 2026Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Modelscited by 0
  12. 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMscited by 0
  13. 2026Realize What Matters: Principled Context Representation for Large-Scale Reasoningcited by 0
  14. 2026Reinforcing Agentic Creativity in Scientific Ideation with Night Sciencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking
  2. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
  3. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
  4. 2025Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory
  5. 2025Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM
  6. 2025Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning
  7. 2025Beyond Pattern Recognition: Probing Mental Representations of LMs
  8. 2025Reasoning LLMs are Wandering Solution Explorers