# Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention

arXiv: https://arxiv.org/abs/2602.18145 · HTML: https://mlsanity.com/paper/2602.18145/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Builds on (most cited first, 26 of 26)

- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [When Attention Sink Emerges in Language Models: An Empirical View](https://mlsanity.com/paper/2410.10781/) (2024)
- [Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps](https://mlsanity.com/paper/2407.07071/) (2024)
- [Discrete Signal Processing on Graphs: Frequency Analysis](https://mlsanity.com/paper/1307.0468/) (2013)
- [On Early Detection of Hallucinations in Factual Question Answering](https://mlsanity.com/paper/2312.14183/) (2023)
- [FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge](https://mlsanity.com/paper/2305.08281/) (2023)
- [The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs](https://mlsanity.com/paper/2504.17768/) (2025)
- [Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching](https://mlsanity.com/paper/2311.15131/) (2023)
- [LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation](https://mlsanity.com/paper/2408.15533/) (2024)
- [Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads](https://mlsanity.com/paper/2505.20045/) (2025)
- [The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States](https://mlsanity.com/paper/2412.17056/) (2024)
- [Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models](https://mlsanity.com/paper/2501.01059/) (2025)
- [A Survey of Automatic Hallucination Evaluation on Natural Language Generation](https://mlsanity.com/paper/2404.12041/) (2024)
- [Hallucinated Span Detection with Multi-View Attention Features](https://mlsanity.com/paper/2504.04335/) (2025)
- [The Pitfalls of Defining Hallucination](https://mlsanity.com/paper/2401.07897/) (2024)
- [LLM Hallucination Detection: A Fast Fourier Transform Method Based on Hidden Layer Temporal Signals](https://mlsanity.com/paper/2509.13154/) (2025)
- [From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models](https://mlsanity.com/paper/2505.00033/) (2025)

## Built on by (most cited first, 2 of 2)

- [When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs](https://mlsanity.com/paper/2605.11559/) (2026)
- [Fingerprinting Multimodal Large Language Models](https://mlsanity.com/paper/2609.20457/) (2026)

## Abstract (arXiv)

Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view of grounding behavior. However, existing approaches typically rely on coarse summaries that fail to capture fine-grained instabilities in attention. Inspired by signal processing, we introduce a frequency-aware perspective on attention by analyzing its variation during generation. We model attention distributions as discrete signals and extract high-frequency components that reflect rapid local changes in attention. Our analysis reveals that hallucinated tokens are associated with high-frequency attention energy, reflecting fragmented and unstable grounding behavior. Based on this insight, we develop a lightweight hallucination detector using high-frequency attention features. Experiments on the RAGTruth and HalluRAG benchmarks show that our approach achieves performance gains over verification-based, internal-representation-based, and attention-based methods across models and tasks.