mlsanityData to 7 Oct 2026

Metacognitive Prompting Improves Understanding in Large Language Models

arXiv 2308.05342 ↗First posted 10 Aug 2023cs.CL cs.AIcited by 58

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In Large Language Models (LLMs), there have been consistent advancements in task-specific performance, largely influenced by effective prompt design. Recent advancements in prompting have enhanced reasoning in logic-intensive tasks for LLMs, yet the nuanced understanding abilities of these models, crucial for processing and interpreting complex information, remain underexplored. In this study, we introduce Metacognitive Prompting (MP), a strategy inspired by human introspective reasoning processes. Using MP, LLMs undergo a systematic series of structured, self-aware evaluations, drawing on both their vast inherent knowledge and new insights. We conduct extensive experiments on four prevalent LLMs: Llama2, PaLM2, GPT-3.5, and GPT-4, across ten natural language understanding (NLU) datasets from GLUE, SuperGLUE, BLUE, and LexGLUE benchmarks. Additionally, we compare our method with chain-of-thought prompting and its advanced versions. The results show that GPT-4 consistently excels across all tasks, while other models have shown significant progress in some tasks when used in conjunction with MP. Furthermore, MP consistently outperforms existing prompting methods in both general and domain-specific NLU tasks. This study underscores the potential to amplify the understanding abilities of LLMs and highlights the benefits of mirroring human introspective reasoning in NLU tasks.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 33 of its 66 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

33 of 66 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  12. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
21 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  5. 2021Ethical and social risks of harm from Language Modelscited by 957
  6. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  7. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  8. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  9. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  10. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  11. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  12. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  13. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  14. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
  15. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  16. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
  17. 2018CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Servicecited by 72
  18. 2023Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understandingcited by 40
  19. 2023An Empirical Study on the Robustness of the Segment Anything Model (SAM)cited by 23
  20. 2022Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsiscited by 14
  21. 2021Empirical Quantitative Analysis of COVID-19 Forecasting Modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 15; 2025: 20; 2026: 18.

58 papers.

Sort:

By area:

  1. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  2. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  3. 2025Agentic Large Language Models, a surveycited by 105
  4. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  5. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  6. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  7. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 39
  8. 2024Self-Reflection in LLM Agents: Effects on Problem-Solving Performancecited by 38
  9. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  10. 2025Ontology Generation using Large Language Modelscited by 23
  11. 2023An evolutionary model of personality traits related to cooperative behavior using a large language modelcited by 18
  12. 2024Training Language Models to Critique With Multi-agent Feedbackcited by 16
38 more
  1. 2023Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Modelscited by 15
  2. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 15
  3. 2024RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Modelscited by 13
  4. 2024Imagining and building wise machines: The centrality of AI metacognitioncited by 12
  5. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 12
  6. 2024Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistantscited by 10
  7. 2025HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimizationcited by 10
  8. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  9. 2024An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guidecited by 7
  10. 2025What Makes a Good Natural Language Prompt?cited by 6
  11. 2025A Survey of Classification Tasks and Approaches for Legal Contractscited by 6
  12. 2024"Let's Argue Both Sides": Argument Generation Can Force Small Models to Utilize Previously Inaccessible Reasoning Capabilitiescited by 4
  13. 2025Schema for In-Context Learningcited by 4
  14. 2026SpecOps: A Fully Automated AI Agent Testing Framework in Real-World GUI Environmentscited by 4
  15. 2024BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Promptscited by 3
  16. 2025IRONIC: Coherence-Aware Reasoning Chains for Multi-Modal Sarcasm Detectioncited by 3
  17. 2025Do Large Vision-Language Models Distinguish between the Actual and Apparent Features of Illusions?cited by 3
  18. 2026Supervising Ralph Wiggum: Exploring a Metacognitive Co-Regulation Agentic AI Loop for Engineering Designcited by 3
  19. 2025Flying Pigs, FaR and Beyond: Evaluating LLM Reasoning in Counterfactual Worldscited by 2
  20. 2025Towards Self-cognitive Exploration: Metacognitive Knowledge Graph Retrieval Augmented Generationcited by 2
  21. 2025Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detectioncited by 2
  22. 2026Fine-Tuning Language Models to Know What They Knowcited by 2
  23. 2026Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Modelscited by 2
  24. 2024Enhancing Reasoning Capacity of SLM using Cognitive Enhancementcited by 1
  25. 2025Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modelingcited by 1
  26. 2026To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Taskscited by 1
  27. 2026MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Modelscited by 1
  28. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  29. 2024AI Predicts AGI: Leveraging AGI Forecasting and Peer Review to Explore LLMs' Complex Reasoning Capabilitiescited by 0
  30. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0
  31. 2025A Pilot Empirical Study on When and How to Use Knowledge Graphs as Retrieval Augmented Generationcited by 0
  32. 2025Nek Minit: Harnessing Pragmatic Metacognitive Prompting for Explainable Sarcasm Detection of Australian and Indian Englishcited by 0
  33. 2025Chain of Methodologies: Scaling Test Time Computation without Trainingcited by 0
  34. 2025SciEvent: Benchmarking Multi-domain Scientific Event Extractioncited by 0
  35. 2025Automatic Minds: Cognitive Parallels Between Hypnotic States and Large Language Model Processingcited by 0
  36. 2026Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answeringcited by 0
  37. 2026Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLMcited by 0
  38. 2026LLMs as annotators of credibility assessment in Danish asylum decisions: evaluating classification performance and errors beyond aggregated metricscited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Meta Reasoning for Large Language Models
  2. 2026Beyond the Prompt in Large Language Models: Comprehension, In-Context Learning, and Chain-of-Thought
  3. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  4. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  5. 2023Re-Reading Improves Reasoning in Large Language Models
  6. 2026Think2: Grounded Metacognitive Reasoning in Large Language Models
  7. 2025DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models
  8. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?