mlsanityData to 14 Oct 2026

No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes

arXiv 2509.10625 ↗First posted 12 Sep 2025cs.CL cs.AIcited by 38

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

Do large language models (LLMs) anticipate when they will answer correctly? To study this, we extract activations after a question is read but before any tokens are generated, and train linear probes to predict whether the model's forthcoming answer will be correct. Across three open-source model families ranging from 7 to 70 billion parameters, projections on this "in-advance correctness direction" trained on generic trivia questions predict success in distribution and on diverse out-of-distribution knowledge datasets, indicating a deeper signal than dataset-specific spurious features, and outperforming black-box baselines and verbalised predicted confidence. Predictive power saturates in intermediate layers and, notably, generalisation falters on questions requiring mathematical reasoning. Moreover, for models responding "I don't know", doing so strongly correlates with the probe score, indicating that the same direction also captures confidence. By complementing previous results on truthfulness and other behaviours obtained with probes and sparse auto-encoders, our work contributes essential findings to elucidate LLM internals.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 36 of its 48 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

36 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Detecting Strategic Deception Using Linear Probes5 Feb

Other references · 35

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  9. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
23 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  3. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  4. 2020Selective Question Answering under Domain Shiftcited by 238
  5. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  6. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  7. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  8. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  9. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  10. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  11. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 79
  12. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 56
  13. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 55
  14. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 40
  15. 2024Finetuning Language Models to Emit Linguistic Expressions of Uncertaintycited by 26
  16. 2024100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instancescited by 24
  17. 2025Benchmarking Deception Probes via Black-to-White Performance Boostscited by 19
  18. 2024Confidence in the Reasoning of Large Language Modelscited by 18
  19. 2025Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Taskscited by 18
  20. 2025Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferencescited by 13
  21. 2025Predicting the Performance of Black-box LLMs through Follow-up Queriescited by 9
  22. 2025Do Sparse Autoencoders Generalize? A Case Study of Answerabilitycited by 8
  23. 2025PredictaBoard: Benchmarking LLM Score Predictabilitycited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 35.

38 papers.

Sort:

By area:

  1. 2026LLMs Encode Their Failures: Predicting Success from Pre-Generation Activationscited by 14
  2. 2025Do Large Language Models Know What They Are Capable Of?cited by 10
  3. 2025Teaching Language Models to Faithfully Express their Uncertaintycited by 7
  4. 2026On Calibration of Large Language Models: From Response To Capabilitycited by 7
  5. 2025Temporal Predictors of Outcome in Reasoning Language Modelscited by 6
  6. 2026Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probingcited by 6
  7. 2026Early Stopping for Large Reasoning Models via Confidence Dynamicscited by 6
  8. 2026Gradient-Stable Attention Heads Signal LLM Correctnesscited by 5
  9. 2026LLM Router: Rethinking Routing with Prefill Activationscited by 5
  10. 2026Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal Statescited by 5
  11. 2026Two Axes of LLM Abstention: Answer Correctness and Question Answerabilitycited by 5
  12. 2026Sparse Reward Subsystem in Large Language Modelscited by 4
26 more
  1. 2026When Role-playing, Do Models Believe What They Say?cited by 4
  2. 2026Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimationcited by 3
  3. 2026Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctnesscited by 3
  4. 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
  5. 2026Correctness-Optimized Residual Activation Lens (CORAL): Transferrable and Calibration-Aware Inference-Time Steeringcited by 2
  6. 2026Testing the Limits of Truth Directions in LLMscited by 2
  7. 2026LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Modelscited by 2
  8. 2026The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representationscited by 1
  9. 2026Capability Self-Assessment in Large Language Modelscited by 1
  10. 2026Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMscited by 1
  11. 2026Latent Programming Horizons in Coding Agentscited by 1
  12. 2026Future Confidence Distillation in Large Language Modelscited by 1
  13. 2026Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scalecited by 1
  14. 2026On the Robustness of LLMs' Internal Representation of Code Correctnesscited by 1
  15. 2026It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectoriescited by 1
  16. 2026Dynamic LLM Routers are Often Misguidedcited by 1
  17. 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Modelscited by 0
  18. 2026Large Language Models Can Take False First Steps at Inference-time Planningcited by 0
  19. 2026Localizing Prompt Ambiguity in Large Language Models with Probe-Targeted Attributioncited by 0
  20. 2026Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steeringcited by 0
  21. 2026Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagationcited by 0
  22. 2026Misaligned Clinical Risk Classification and Cost Asymmetry in Open-Weight Large Language Modelscited by 0
  23. 2026The Answer-Basin Representation Hypothesis: We Are Not Probing or Steering Conceptscited by 0
  24. 2026The Low-Rank Structure of VLA Reinforcement Learningcited by 0
  25. 2026Cross-lingual Calibration of Pre-Generation Success Probes for Multilingual LLM Routingcited by 0
  26. 2026Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy
  2. 2025Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth
  3. 2025Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs
  4. 2025Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident, Especially When They are Wrong
  5. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models
  6. 2026LLMs Should Express Uncertainty Explicitly
  7. 2025Trained on Tokens, Calibrated on Concepts: The Emergence of Semantic Calibration in LLMs
  8. 2025Inside-Out: Hidden Factual Knowledge in LLMs