Quickest Detection of Hallucination Onset: Delay Bounds and Learned CUSUM Statistics
Not on the map. Why?
Abstract (arXiv)
Token-level hallucination detectors are evaluated as classifiers, by AUC over all tokens, yet a streaming monitor is judged by its reaction time: the number of tokens that pass between the onset of a hallucination and the alarm. We formulate hallucination onset detection as a quickest change detection problem. A first-order Markov model of the latent faithful/hallucinated state, validated on RAGTruth, places the task inside classical change-point theory and yields Lorden's lower bound on detection delay: about 1.3 tokens at a false-alarm rate of 0.01. We then show that a causal recurrent labeler acts as a CUSUM with a learned increment. Among the onsets it catches it detects in 11-13 tokens, against 31 for a linear per-token baseline, though at this false-alarm budget every detector catches under a third of onsets and the recall-honest delay is 56-66 tokens: low-false-alarm onset detection is hard. A controlled decomposition attributes the speed advantage mostly to a better per-token score rather than to temporal accumulation. An information-rate optimality theorem of Donsker-Varadhan type explains the remaining order-of-magnitude gap: the learned score realizes only 1/4.5 of the divergence the features carry, a deficit that recalibration cannot remove, with the remainder a finite-horizon effect. Classification metrics conceal this delay structure; sequential analysis makes it measurable.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 29 references that are in the map. First posted on arXiv 10 Jun 2026.
Builds on
12 of 29 references.
References · 12
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
- 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 138
- 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 34
- 2026HALT: Hallucination Assessment via Log-probs as Time seriescited by 8
- 2022Neural network-based CUSUM for online change-point detectioncited by 5
- 2022Recurrent Neural Networks and Universal Approximation of Bayesian Filterscited by 5
- 2025First Hallucination Tokens Are Different from Conditional Onescited by 4
- 2026Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generationcited by 2
- 2026Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometrycited by 2
- 2026Sequential statistical inference for Large Language Models: Representation, validity, and monitoringcited by 2
- 2020Exactly Optimal Bayesian Quickest Change Detection for Hidden Markov Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.