Locating Answer-Correctness Signals in Frozen Large Language Models
Not on the map. Why?
Abstract (arXiv)
Language models expose internal signals that predict whether an answer is correct, readable from a single forward pass of a frozen model without additional generations. Yet existing probes often commit to one signal family or layer and can be brittle under distribution shift; in retrieval-augmented settings, many specialized detectors instead target passage faithfulness, which can diverge from correctness when retrieved evidence is unhelpful or conflicting. We therefore ask where answer correctness is readable, which internal signal families carry it, and how they should be combined. We search over hidden states, token probabilities, residual-stream features, attention, and their fusion, treating the selected readouts as a predictive measurement rather than a mechanistic localization. We run this analysis separately in closed-book and with-context settings, since context can change which readouts are informative. A consistent anatomy emerges: correctness concentrates in the answer span, recovered from the answer tokens even under retrieval, and the families carry it complementarily, so fusing them helps most out of distribution, where a single signal is weakest. The protocol is effective across two backbones and gates a retrieval controller as one downstream use.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 39 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
35 of 39 references.
References · 35
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
- 2023The Internal State of an LLM Knows When It's Lyingcited by 644
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
- 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
23 more
- 2024Knowledge Conflicts for LLMs: A Surveycited by 274
- 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
- 2026Gemma 4 Technical Reportcited by 187
- 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
- 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 100
- 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
- 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 84
- 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
- 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 42
- 2024SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generationcited by 40
- 2025ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMscited by 31
- 2020Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?cited by 29
- 2024Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrievalcited by 28
- 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
- 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signalscited by 8
- 2025The Missing Parts: Augmenting Fact Verification with Half-Truth Detectioncited by 7
- 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
- 2026MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probingcited by 5
- 2026Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graphcited by 2
- 2026Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Makingcited by 2
- 2026EnsemHalDet: Robust VLM Hallucination Detection via Ensemble of Internal State Detectorscited by 1
- 2026Debating the Unspoken: Role-Anchored Multi-Agent Reasoning for Half-Truth Detectioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
- 2026Language-Conditioned Token and Reasoning Efficiency in Large Language Models: A Paired Cross-Lingual Study Protocol
- 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
- 2026When are likely answers right? On Sequence Probability and Correctness in LLMs
- 2026How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Models
- 2026Grounding Without Corrective Control: Truth-Tracking Profiles for Large Language Models
- 2026When Does a Language Model Commit? A Finite-Answer Theory of Pre-Verbalization Commitment
- 2025Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs