# Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits

arXiv: https://arxiv.org/abs/2512.20578 · HTML: https://mlsanity.com/paper/2512.20578/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 23 Dec 2025.

## Builds on (most cited first, 22 of 22)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Lessons of Developing Process Reward Models in Mathematical Reasoning](https://mlsanity.com/paper/2501.07301/) (2025)
- [MathArena: Evaluating LLMs on Uncontaminated Math Competitions](https://mlsanity.com/paper/2505.23281/) (2025)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://mlsanity.com/paper/2412.06559/) (2024)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy](https://mlsanity.com/paper/2507.01352/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [RewardBench 2: Advancing Reward Model Evaluation](https://mlsanity.com/paper/2506.01937/) (2025)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [HelpSteer2-Preference: Complementing Ratings with Preferences](https://mlsanity.com/paper/2410.01257/) (2024)
- [Efficient Test-Time Scaling via Self-Calibration](https://mlsanity.com/paper/2503.00031/) (2025)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation](https://mlsanity.com/paper/2410.13640/) (2024)
- [Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States](https://mlsanity.com/paper/2402.09733/) (2024)
- [Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence](https://mlsanity.com/paper/2505.20325/) (2025)
- [Self-Evaluation of Large Language Model based on Glass-box Features](https://mlsanity.com/paper/2403.04222/) (2024)

## Built on by (most cited first, 14 of 14)

- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought](https://mlsanity.com/paper/2603.18940/) (2026)
- [LLM Router: Rethinking Routing with Prefill Activations](https://mlsanity.com/paper/2603.20895/) (2026)
- [WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Tracking](https://mlsanity.com/paper/2603.27343/) (2026)
- [Uncertainty Propagation in LLM-Based Systems](https://mlsanity.com/paper/2604.23505/) (2026)
- [Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making](https://mlsanity.com/paper/2607.14277/) (2026)
- [How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Models](https://mlsanity.com/paper/2608.02089/) (2026)
- [Trust or Abstain? A Self-Aware RAG Approach](https://mlsanity.com/paper/2605.18792/) (2026)
- [ARBITER: Reasoning Trajectory Basins and Majority Vote Failures in Test-Time Sampling](https://mlsanity.com/paper/2605.26172/) (2026)
- [Dynamic LLM Routers are Often Misguided](https://mlsanity.com/paper/2610.02762/) (2026)
- [Structural Rigidity and the 57-Token Predictive Window: A Physical Framework for Inference-Layer Governability in Large Language Models](https://mlsanity.com/paper/2604.03524/) (2026)
- [Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models](https://mlsanity.com/paper/2605.01853/) (2026)
- [Nudgeability: Reasoning Models Follow Confidence Signals Without Tracking Their Own Competence](https://mlsanity.com/paper/2609.34572/) (2026)
- [LatentSift: Policy-State Filtering for Token-Efficient Verification of Software Engineering Agents](https://mlsanity.com/paper/2609.36371/) (2026)

## Abstract (arXiv)

Large language models (LLMs) generate fluent and complex outputs but often fail to recognize their own mistakes and hallucinations. Existing approaches typically rely on external judges, multi-sample consistency, or text-based self-critique, which incur additional compute or correlate weakly with true correctness. We ask: can LLMs predict their own failures by inspecting internal states during inference? We introduce Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to perform intrinsic self-verification by decoding signals from hidden states and attention patterns. Gnosis passively observes internal traces, compresses them into fixed-budget descriptors, and predicts correctness with negligible inference cost, adding only ~5M parameters and operating independently of sequence length. Across math reasoning, open-domain question answering, and academic knowledge benchmarks, and over frozen backbones ranging from 1.7B to 20B parameters, Gnosis consistently outperforms strong internal baselines and large external judges in both accuracy and calibration. Moreover, it generalizes zero-shot to partial generations, enabling early detection of failing trajectories and compute-aware control. These results show that reliable correctness cues are intrinsic to generation process and can be extracted efficiently without external supervision.