The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring
Not on the map. Why?
Abstract (arXiv)
We introduce a cross-domain behavioural assay of monitoring-control coupling in LLMs, grounded in the Nelson and Narens (1990) metacognitive framework and applying human psychometric methodology to LLM evaluation. The battery comprises 524 items across six cognitive domains (learning, metacognitive calibration, social cognition, attention, executive function, prospective regulation), each grounded in an established experimental paradigm. Tasks T1-T5 were pre-registered on OSF prior to data collection; T6 was added as an exploratory extension. After every forced-choice response, dual probes adapted from Koriat and Goldsmith (1996) ask the model to KEEP or WITHDRAW its answer and to BET or decline. The critical metric is the withdraw delta: the difference in withdrawal rate between incorrect and correct items. Applied to 20 frontier LLMs (10,480 evaluations), the battery discriminates three profiles consistent with the Nelson-Narens architecture: blanket confidence, blanket withdrawal, and selective sensitivity. Accuracy rank and metacognitive sensitivity rank are largely inverted. Retrospective monitoring and prospective regulation appear dissociable (r = .17, 95% CI wide given n=20; exemplar-based evidence is the primary support). Scaling on metacognitive calibration is architecture-dependent: monotonically decreasing (Qwen), monotonically increasing (GPT-5.4), or flat (Gemma). Behavioural findings converge structurally with an independent Type-2 SDT approach, providing preliminary cross-method construct validity. All items, data, and code: https://github.com/synthiumjp/metacognitive-monitoring-battery.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 4 of its 16 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
4 of 16 references.
References · 4
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2026Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theorycited by 6
- 2026Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Modelscited by 4
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Reportcited by 2
- 2026Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screencited by 2
- 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
- 2026Domain-level metacognitive monitoring in frontier LLMs: A 33-model atlascited by 0
- 2026Reality Monitoring in Large Language Models: Self-Knowledge That Transforms with Conversation Memorycited by 0
- 2026Reality Monitoring in Large Language Models: Self-Knowledge That Transforms with Conversation Memory27 Jul 2026
- 2026Metacognition in LLMs: Foundations, Progress, and Opportunities13 Jul 2026
- 2026Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen24 Apr 2026
- 2026Domain-level metacognitive monitoring in frontier LLMs: A 33-model atlas21 Apr 2026
- 2026Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Report20 Apr 2026