mlsanityData to 14 Oct 2026

Metacognition in LLMs: Foundations, Progress, and Opportunities

arXiv 2607.11881 ↗First posted 13 Jul 2026cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 179 of its 317 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

179 of 317 references.

References · 179

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  5. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  9. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  10. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  11. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  12. 2024Alignment faking in large language modelscited by 360
38 more
  1. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  2. 2023Poisoning Language Models During Instruction Tuningcited by 269
  3. 2023Do Large Language Models Know What They Don't Know?cited by 243
  4. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  5. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  6. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 122
  7. 2024To Believe or Not to Believe Your LLMcited by 120
  8. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  9. 2021Truthful AI: Developing and governing AI that does not liecited by 113
  10. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 113
  11. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 112
  12. 2024What Large Language Models Know and What People Think They Knowcited by 110
  13. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
  14. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  15. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  16. 2025Tell me about yourself: LLMs are aware of their learned behaviorscited by 91
  17. 2023The Metacognitive Demands and Opportunities of Generative AIcited by 87
  18. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  19. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 84
  20. 2024On Verbalized Confidence Scores for LLMscited by 83
  21. 2026Emergent Introspective Awareness in Large Language Modelscited by 82
  22. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 79
  23. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 72
  24. 2025Are Reasoning Models More Prone to Hallucination?cited by 64
  25. 2024Escalation Risks from Language Models in Military and Diplomatic Decision-Makingcited by 63
  26. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 62
  27. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 62
  28. 2024LegalAgentBench: Evaluating LLM Agents in Legal Domaincited by 59
  29. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  30. 2024Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performancecited by 58
  31. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  32. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 55
  33. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  34. 2025Evaluating Large Language Models in Scientific Discoverycited by 42
  35. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 41
  36. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 41
  37. 2025Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokenscited by 41
  38. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 40

50 most cited of 179 All 179 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Understanding Cognition-Induced Risks in Agentic AI Systemscited by 0