mlsanityData to 7 Oct 2026

Imagining and building wise machines: The centrality of AI metacognition

arXiv 2411.02478 ↗First posted 4 Nov 2024cs.AI cs.CY cs.HCcited by 12

Not on the map. Why?

Abstract (arXiv)

Although AI has become increasingly smart, its wisdom has not kept pace. In this article, we examine what is known about human wisdom and sketch a vision of its AI counterpart. We analyze human wisdom as a set of strategies for solving intractable problems-those outside the scope of analytic techniques-including both object-level strategies like heuristics [for managing problems] and metacognitive strategies like intellectual humility, perspective-taking, or context-adaptability [for managing object-level strategies]. We argue that AI systems particularly struggle with metacognition; improved metacognition would lead to AI more robust to novel environments, explainable to users, cooperative with others, and safer in risking fewer misaligned goals with human users. We discuss how wise AI might be benchmarked, trained, and implemented.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 31 of its 131 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

31 of 131 references.

References · 31

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  6. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  7. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  8. 2024Large Language Models: A Surveycited by 506
  9. 2018AI safety via debatecited by 384
  10. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  11. 2023AI Alignment: A Comprehensive Surveycited by 256
  12. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
19 more
  1. 2020Open Problems in Cooperative AIcited by 167
  2. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  3. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
  4. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 82
  5. 2024Building Machines that Learn and Think with Peoplecited by 73
  6. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  7. 2021People construct simplified mental representations to plancited by 45
  8. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  9. 2024Metacognitive Retrieval-Augmented Large Language Modelscited by 26
  10. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  11. 2013Reasoning About Beliefs and Actions Under Computational Resource Constraintscited by 18
  12. 2019Towards a Quantum-Like Cognitive Architecture for Decision-Makingcited by 17
  13. 2024Metacognition is all you need? Using Introspection in Generative Agents to Improve Goal-directed Behaviorcited by 15
  14. 2024Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approachcited by 11
  15. 2023OpenCog Hyperon: A Framework for AGI at the Human Level and Beyondcited by 9
  16. 2024Can ChatGPT Make Explanatory Inferences? Benchmarks for Abductive Reasoningcited by 6
  17. 2024Metacognitive Myopia in Large Language Modelscited by 6
  18. 2024Generative AI as a metacognitive agent: A comparative mixed-method study with human participants on ICF-mimicking exam performancecited by 4
  19. 2023Metacognition-Enhanced Few-Shot Prompting With Positive Reinforcementcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 4.

12 papers.

  1. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Modelscited by 31
  2. 2025Agentic AI Needs a Systems Theorycited by 19
  3. 2025Truly Self-Improving Agents Require Intrinsic Metacognitive Learningcited by 13
  4. 2025Evaluating Long-Term Memory for Long-Context Question Answeringcited by 8
  5. 2025Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?cited by 4
  6. 2025Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environmentscited by 4
  7. 2025Contemplative Artificial Intelligencecited by 1
  8. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  9. 2025Realist and Pluralist Conceptions of Intelligence and Their Implications on AI Researchcited by 0
  10. 2026Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AIcited by 0
  11. 2026The Biomimetic Architecture of Software 4.0cited by 0
  12. 2026Psychological Competence as a Missing Dimension in AI Evaluationcited by 0