Knowing Is Not Choosing: What Explicit Verification Adds Beyond Generative Preference
Not on the map. Why?
Abstract (arXiv)
Generating a correct answer does not mean that a language model will select it. We separate factual recall into three steps: generating a correct candidate, ranking the available candidates, and selecting the final answer. Pre-generation readouts predict factual recall and which questions sampling will cover across three model families, but say little about whether an available correct answer will ultimately be selected. Explicit verification with P(True) improves within-question ranking over mean log-likelihood in Gemma, Qwen3, and Llama, with AUROC gains of 0.08--0.12. In a prospectively defined Gemma cohort, verification raises plurality accuracy by about 5 points, and still gains about 2 points over chat-template likelihood, a stronger generative baseline. The advantage is strongest for relations with common-answer priors and depends on access to the entity; masking the entity removes the ranking advantage in larger Qwen models. Finally, the measured benefit depends on how correctness is defined: recall-oriented reference matching can credit option lists favored by likelihood and substantially understate the improvement seen under human semantic judgments. Prior work shows that models can carry latent factual knowledge and judge candidate answers; we show that these capabilities do not collapse into a single notion of ``knowing,'' and trace where information is gained, lost, or mismeasured between availability, ranking, and final choice.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 50 references that are in the map. First posted on arXiv 27 Sep 2026.
Builds on
28 of 50 references.
References · 28
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
16 more
- 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
- 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
- 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
- 2025Confidence Improves Self-Consistency in LLMscited by 111
- 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
- 2024The Limits of Inference Scaling Through Resamplingcited by 51
- 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
- 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 39
- 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 33
- 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 15
- 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 9
- 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 7
- 2024Keep Guessing? When Considering Inference Scaling, Mind the Baselinescited by 3
- 2026PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifactscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Unsure but Certain: Uncovering the Representation-Confidence Gap in Diffusion Language Models
- 2026An Analysis of Training-Free Self-Reported Confidence in Language Models
- 2022Teaching language models to support answers with verified quotes
- 2026Locating Answer-Correctness Signals in Frozen Large Language Models
- 2026Learning When to Trust via Selective Context Preference Optimization
- 2026Certainty Is Not Just Correctness: Rethinking Token-Level Certainty in LLM Reasoning
- 2026Language-Conditioned Token and Reasoning Efficiency in Large Language Models: A Paired Cross-Lingual Study Protocol
- 2026Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation