# Confidently Wrong: Detecting Hallucinations in Financial Question Answering from LLM Internal States

arXiv: https://arxiv.org/abs/2607.11414 · HTML: https://mlsanity.com/paper/2607.11414/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [Building Production-Ready Probes For Gemini](https://mlsanity.com/paper/2601.11516/) (2026)
- [Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks](https://mlsanity.com/paper/2601.04603/) (2026)
- [The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs](https://mlsanity.com/paper/2508.08285/) (2025)
- [FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models](https://mlsanity.com/paper/2507.20930/) (2025)
- [HACK: Hallucinations Along Certainty and Knowledge Axes](https://mlsanity.com/paper/2510.24222/) (2025)
- [PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts](https://mlsanity.com/paper/2605.17028/) (2026)
- [Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs](https://mlsanity.com/paper/2606.02628/) (2026)
- [K-FinHallu: A Hallucination Detection Benchmark for Multi-Turn RAG in Korean Finance](https://mlsanity.com/paper/2605.29523/) (2026)

## Abstract (arXiv)

Large language models (LLMs) in financial applications fail most consequentially when they are confidently wrong. Hedged, uncertain answers invite scrutiny, whereas confident errors silently degrade downstream decisions without warning. We ask how reliably such confidently wrong answers, or confident hallucinations, can be detected from a model's internal activations, and whether those activations carry information beyond its output-level confidence signals. We train linear probes on the residual stream and evaluate them on two established numerical question-answering (QA) benchmarks built from real filings, FinQA and TAT-QA. Behavioral confidence is measured as the agreement among eight resampled answers to the same question, and probe effectiveness is compared against baselines, such as token log-probabilities and the model's own True/False self-assessment of its answer. Our findings show that among confident answers, those for which all eight resamples agree, 15-23% are wrong on FinQA. There the probes have a significant advantage over output-level baselines in detecting hallucinations, holding 0.68-0.77 AUROC while the best baselines fall to 0.55-0.63, across Qwen3-8B, Llama-3.1-8B, and Gemma-2-9B. Our results suggest that probing can be a cost-effective triage mechanism for routing LLM answers to human review and quality control procedures in high-stakes financial applications. Code and data are available at https://github.com/richardzhewang/confidently-wrong.