From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction
Not on the map. Why?
Abstract (arXiv)
We test whether a geometric hallucination taxonomy -- classifying failures as center-drift (Type~1), wrong-well convergence (Type~2), or coverage gaps (Type~3) -- can distinguish hallucination types through controlled induction in GPT-2. Using a two-level statistical design with prompts (N = 15/group) as the unit of inference, we run each experiment 20 times with different generation seeds to quantify result stability. In static embeddings, Type~3 norm separation is robust (significant in 18/20 runs, Holm-corrected in 14/20, median r = +0.61). In contextual hidden states, the Type~3 norm effect direction is stable (19/20 runs) but underpowered at N = 15 (significant in 4/20, median r = -0.28). Types~1 and~2 do not separate in either space (≤,3/20 runs). Token-level tests inflate significance by 4--16× through pseudoreplication -- a finding replicated across all 20 runs. The results establish coverage-gap hallucinations as the most geometrically distinctive failure mode, carried by magnitude rather than direction, and confirm the Type~1/2 non-separation as genuine at 124M parameters.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 27 Feb 2026.
Builds on
9 of 21 references.
References · 9
- 2017Attention Is All You Needcited by 59,932
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
- 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
- 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
- 2026Detecting LLM Hallucinations via Embedding Cluster Geometry: A Three-Type Taxonomy with Measurable Signaturescited by 2
Built on by
Not cited yet.