mlsanityData to 7 Oct 2026

What do Geometric Hallucination Detection Metrics Actually Measure?

arXiv 2602.09158 ↗First posted 9 Feb 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Hallucination remains a barrier to deploying generative models in high-consequence applications. This is especially true in cases where external ground truth is not readily available to validate model outputs. This situation has motivated the study of geometric signals in the internal state of an LLM that are predictive of hallucination and require limited external knowledge. Given that there are a range of factors that can lead model output to be called a hallucination (e.g., irrelevance vs incoherence), in this paper we ask what specific properties of a hallucination these geometric statistics actually capture. To assess this, we generate a synthetic dataset which varies distinct properties of output associated with hallucination. This includes output correctness, confidence, relevance, coherence, and completeness. We find that different geometric statistics capture different types of hallucinations. Along the way we show that many existing geometric detection methods have substantial sensitivity to shifts in task domain (e.g., math questions vs. history questions). Motivated by this, we introduce a simple normalization method to mitigate the effect of domain shift on geometric statistics, leading to AUROC gains of +34 points in multi-domain settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

15 of 18 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  7. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  9. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  10. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  11. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  12. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
3 more
  1. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 49
  2. 2024Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimensioncited by 45
  3. 2024On the Universal Truthfulness Hyperplane Inside LLMscited by 23

Built on by

Not cited yet.