mlsanityData to 14 Oct 2026

Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AI

arXiv 2605.23981 ↗First posted 13 May 2026q-bio.NC cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative AI research increasingly confronts a shared problem: systems must sustain yet govern their own generative activity when uncertainty is high, evidence is missing, or context is insufficient. This position paper argues that metacognition should become the scientific framework for bounded and effective self governance in generative AI, where output generation is properly evaluated together with the capacities through which generative systems navigate and regulate their own activity. We advance this position by showing that bounded and effective AI self-governance requires metacognitive alignment across computational, algorithmic, and ecological levels. At the computational level, metacognition specifies the meta-level functions a system is meant to serve, such as monitoring, evaluation, control, and adaptation. At the algorithmic level, these functions are realized through procedures such as elicitation, iteration, and modularization. At the ecological level, metacognitive signals become meaningful, actionable, and accountable within the interface, workflow, and accountability arrangements. Metacognition thus makes it possible to conceive generative AI as both capable and well-governed, rather than treating capability and governance as competing aims.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 105 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

43 of 105 references.

References · 43

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  10. 2018Datasheets for Datasetscited by 1,386
  11. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  12. 2018Model Cards for Model Reportingcited by 1,080
31 more
  1. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  2. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  3. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  4. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  5. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  6. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  7. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  9. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  10. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  11. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
  12. 2018Actionable Recourse in Linear Classificationcited by 328
  13. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  14. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  15. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  16. 2019A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cyclecited by 193
  17. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  18. 2020Algorithmic recourse under imperfect causal knowledge: a probabilistic approachcited by 140
  19. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation Systemcited by 139
  20. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  21. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  22. 2023System 2 Attention (is something you might need too)cited by 89
  23. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 76
  24. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  25. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 41
  26. 2024Metacognitive Retrieval-Augmented Large Language Modelscited by 26
  27. 2020Levels of Analysis for Machine Learningcited by 11
  28. 2025Levels of Analysis for Large Language Modelscited by 10
  29. 2025The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Toolscited by 6
  30. 2025A Dynamical Systems Perspective on the Analysis of Neural Networkscited by 3
  31. 2026How AI Aggregation Affects Knowledgecited by 2

Built on by

Not cited yet.