# Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AI

arXiv: https://arxiv.org/abs/2605.15567 · HTML: https://mlsanity.com/paper/2605.15567/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 102 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 30 of 36)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [The Hidden Vulnerability of Distributed Learning in Byzantium](https://mlsanity.com/paper/1802.07927/) (2018)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- [Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving](https://mlsanity.com/paper/2405.12205/) (2024)
- [Fast and Slow Learning of Recurrent Independent Mechanisms](https://mlsanity.com/paper/2105.08710/) (2021)
- [Trading Off Privacy, Utility and Efficiency in Federated Learning](https://mlsanity.com/paper/2209.00230/) (2022)
- [Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars](https://mlsanity.com/paper/1707.00051/) (2017)
- [Reasoning About Beliefs and Actions Under Computational Resource Constraints](https://mlsanity.com/paper/1304.2759/) (2013)
- [Introspective Perception: Learning to Predict Failures in Vision Systems](https://mlsanity.com/paper/1607.08665/) (2016)
- [Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction](https://mlsanity.com/paper/2210.03067/) (2022)
- [Imagining and building wise machines: The centrality of AI metacognition](https://mlsanity.com/paper/2411.02478/) (2024)
- [Metacognitive AI: Framework and the Case for a Neurosymbolic Approach](https://mlsanity.com/paper/2406.12147/) (2024)
- [Rule-Based Error Detection and Correction to Operationalize Movement Trajectory Classification](https://mlsanity.com/paper/2308.14250/) (2023)
- [Error Detection and Constraint Recovery in Hierarchical Multi-Label Classification without Prior Knowledge](https://mlsanity.com/paper/2407.15192/) (2024)
- [Failures in Perspective-taking of Multimodal AI Systems](https://mlsanity.com/paper/2409.13929/) (2024)
- [Towards Machine Wald](https://mlsanity.com/paper/1508.02449/) (2015)
- [Gluing Neural Networks Symbolically Through Hyperdimensional Computing](https://mlsanity.com/paper/2205.15534/) (2022)
- [Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environments](https://mlsanity.com/paper/2505.19361/) (2025)

## Abstract (arXiv)

This position paper argues for metacognition as a general design principle for creating more accurate, secure, and efficient AI. The metacognitive solution involves systems monitoring their own states and judiciously allocating resources depending on each problem instance's difficulty or cost of mistakes. Drawing inspiration both from past work on resource-rational AI and from well-documented metacognitive strategies in psychology and cognitive science, we identify specific challenges in embedding these strategies into AI design and highlight open theoretical and implementation problems. We showcase these principles through a tangible example of improved learning efficiency, effectiveness, and security in a Federated Learning (FL) case study. We show how these principles can be translated into practice with a novel software framework developed specifically to allow the community to design, deploy, and experiment with metacognition-enabled AI applications.