mlsanityData to 14 Oct 2026

Hallucination Detection in LLMs Using Spectral Features of Attention Maps

arXiv 2502.17598 ↗First posted 24 Feb 2025cs.LG cs.AI cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable performance across various tasks but remain prone to hallucinations. Detecting hallucinations is essential for safety-critical applications, and recent methods leverage attention map properties to this end, though their effectiveness remains limited. In this work, we investigate the spectral features of attention maps by interpreting them as adjacency matrices of graph structures. We propose the LapEigvals method, which utilises the top-k eigenvalues of the Laplacian matrix derived from the attention maps as an input to hallucination detection probes. Empirical evaluations demonstrate that our approach achieves state-of-the-art hallucination detection performance among attention-based methods. Extensive ablation studies further highlight the robustness and generalisation of LapEigvals, paving the way for future advancements in the hallucination detection domain.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 22 of its 50 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

22 of 50 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  9. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  10. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  11. 2018CoQA: A Conversational Question Answering Challengecited by 683
  12. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
10 more
  1. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 478
  2. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  3. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  4. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  5. 2023Understanding Oversquashing in GNNs through the Lens of Effective Resistancecited by 97
  6. 2024Transformers need glasses! Information over-squashing in language taskscited by 91
  7. 2024On the Role of Attention Masks and LayerNorm in Transformerscited by 48
  8. 2024Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Modelscited by 15
  9. 2024PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamicscited by 14
  10. 2024FactLens: Benchmarking Fine-Grained Fact Verificationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 34.

43 papers.

Sort:

By area:

  1. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  2. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 19
  3. 2025Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Modelscited by 14
  4. 2026Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systemscited by 12
  5. 2025EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMscited by 10
  6. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 6
  7. 2026Gradient-Stable Attention Heads Signal LLM Correctnesscited by 5
  8. 2026MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probingcited by 5
  9. 2026Collective Hallucination in Multi-Agent LLMs:Modeling and Defensecited by 5
  10. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  11. 2025Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processescited by 3
  12. 2026HalluZig: Hallucination Detection using Zigzag Persistencecited by 2
31 more
  1. 2026Spectral Guardrails for Agents in the Wild: Detecting Tool Use Hallucinations via Attention Topologycited by 2
  2. 2026CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?cited by 2
  3. 2025Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMscited by 1
  4. 2026Sanity Checks for Long-Form Hallucination Detectioncited by 1
  5. 2026Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scalecited by 1
  6. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinationscited by 0
  7. 2026Spectral Archaeology: The Causal Topology of Model Evolutioncited by 0
  8. 2026Listen to the Layers: Mitigating Hallucinations with Inter-Layer Disagreementcited by 0
  9. 2026VIGIL: Tackling Hallucination Detection in Image Recontextualizationcited by 0
  10. 2026UGID: Unified Graph Isomorphism for Debiasing Large Language Modelscited by 0
  11. 2026Attention Sinks as Internal Signals for Hallucination Detection in Large Language Modelscited by 0
  12. 2026CausalGaze: Unveiling Hallucinations via Counterfactual Graph Intervention in Large Language Modelscited by 0
  13. 2026Geometric Decoupling: Diagnosing the Structural Instability of Latentcited by 0
  14. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgmentscited by 0
  15. 2026Self-Attention as Transport: Limits of Symmetric Spectral Diagnosticscited by 0
  16. 2026Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signalscited by 0
  17. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generationcited by 0
  18. 2026KG-Guard: Graph-Based Hallucination Detection for Knowledge Base Question Answeringcited by 0
  19. 2026Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Modelscited by 0
  20. 2026Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steeringcited by 0
  21. 2026Interpretability-Guided Soft Pruning of Attention Heads in Vision Transformerscited by 0
  22. 2026A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learningcited by 0
  23. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  24. 2026Investigating Linear Probe Robustness to Linguistic Register, Medical Specialty, and Corpus Shifts in Medical QAcited by 0
  25. 2026CodeTD: Topology of Attention Detects Hallucinations in Code LLMscited by 0
  26. 2026NeuroActiSep: Detecting Factual Hallucinations from Feed-Forward Neurons in a Single Passcited by 0
  27. 2026When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMscited by 0
  28. 2026Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharingcited by 0
  29. 2026Semantic Uncertainty Quantification Needs Factual Equivalencecited by 0
  30. 2026Locating Answer-Correctness Signals in Frozen Large Language Modelscited by 0
  31. 2026Spectral Geometry of Attention: From Information Routing to Uncertaintycited by 0