# Pre-Generation Hallucination Detection in Large Language Models via Soft-Target Attention Probing

arXiv: https://arxiv.org/abs/2606.21917 · HTML: https://mlsanity.com/paper/2606.21917/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 20 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps](https://mlsanity.com/paper/2407.07071/) (2024)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [LLM Internal States Reveal Hallucination Risk Faced With a Query](https://mlsanity.com/paper/2407.03282/) (2024)
- [Do Androids Know They're Only Dreaming of Electric Sheep?](https://mlsanity.com/paper/2312.17249/) (2023)
- [A Unified Analysis of Stochastic Momentum Methods for Deep Learning](https://mlsanity.com/paper/1808.10396/) (2018)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [Estimating Knowledge in Large Language Models Without Generating a Single Token](https://mlsanity.com/paper/2406.12673/) (2024)
- [Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home](https://mlsanity.com/paper/2501.12835/) (2025)
- [Hallucination Detection in LLMs with Topological Divergence on Attention Graphs](https://mlsanity.com/paper/2504.10063/) (2025)
- [HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token](https://mlsanity.com/paper/2603.05465/) (2026)
- [Deep Neural Network Ensembles for Time Series Classification](https://mlsanity.com/paper/1903.06602/) (2019)
- [FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMs](https://mlsanity.com/paper/2410.02899/) (2024)

## Built on by (most cited first, 1 of 1)

- [HalluTracer: Pre-Decoding Truthfulness Prediction via Depth-Averaged Probe-Logit](https://mlsanity.com/paper/2608.16353/) (2026)

## Abstract (arXiv)

Detecting hallucination risk before generation enables abstention, retrieval augmentation, and routing decisions without incurring the cost of decoding. While prior work has shown that such risk can be estimated from a model's internal representations, existing approaches treat this as binary classification over a single decoded output. We instead formulate it as a risk-estimation problem. Under this formulation, we introduce soft-target supervision based on the empirical answer error rate over stochastically sampled outputs - an estimator we prove to be the unique unbiased minimum-variance estimator of the model's per-prompt error probability under its sampling distribution. We further adapt attention probing to the pre-generation setting, enabling the detector to selectively aggregate hallucination-relevant prompt representations. Across three question-answering benchmarks and five models, attention probing outperforms linear probing on short-answer tasks. Replacing binary labels with soft-target supervision further and consistently improves detection quality.