Whitening Reveals Cluster Commitment as the Geometric Separator of Hallucination Types
Not on the map. Why?
Abstract (arXiv)
A geometric hallucination taxonomy distinguishes three failure types -- center-drift (Type~1), wrong-well convergence (Type~2), and coverage gaps (Type~3) -- by their signatures in embedding cluster space. Prior work found Types~1 and~2 indistinguishable in full-dimensional contextual measurement. We address this through PCA-whitening and eigenspectrum decomposition on GPT-2-small, using multi-run stability analysis (20 seeds) with prompt-level aggregation. Whitening transforms the micro-signal regime into a space where peak cluster alignment (maxsim) separates Type~2 from Type~3 at Holm-corrected significance, with condition means following the taxonomy's predicted ordering: Type~2 (highest commitment) > Type~1 (intermediate) > Type~3 (lowest). A first directionally stable but underpowered hint of Type~1/2 separation emerges via the same metric, generating a capacity prediction for larger models. Prompt diversification from 15 to 30 prompts per group eliminates a false positive in whitened entropy that appeared robust at the smaller set, demonstrating prompt-set sensitivity in the micro-signal regime. Eigenspectrum decomposition localizes this artifact to the dominant principal components and confirms that Type~1/2 separation does not emerge in any spectral band, rejecting the spectral mixing hypothesis. The contribution is threefold: whitening as preprocessing that reveals cluster commitment as the theoretically correct separating metric, evidence that the Type~1/2 boundary is a capacity limitation rather than a measurement artifact, and a methodological finding about prompt-set fragility in near-saturated representation spaces.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 32 references that are in the map. First posted on arXiv 8 Mar 2026.
Builds on
13 of 32 references.
References · 13
- 2017Attention Is All You Needcited by 59,932
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
- 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
- 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
Built on by
Not cited yet.