# Hallucinated Span Detection with Multi-View Attention Features

arXiv: https://arxiv.org/abs/2504.04335 · HTML: https://mlsanity.com/paper/2504.04335/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 14 of 14)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [On Hallucination and Predictive Uncertainty in Conditional Language Generation](https://mlsanity.com/paper/2103.15025/) (2021)
- [Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps](https://mlsanity.com/paper/2407.07071/) (2024)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [Do Attention Heads in BERT Track Syntactic Dependencies?](https://mlsanity.com/paper/1911.12246/) (2019)
- [LLM Internal States Reveal Hallucination Risk Faced With a Query](https://mlsanity.com/paper/2407.03282/) (2024)
- [TENER: Adapting Transformer Encoder for Named Entity Recognition](https://mlsanity.com/paper/1911.04474/) (2019)
- [Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer](https://mlsanity.com/paper/2205.11631/) (2022)
- [Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation](https://mlsanity.com/paper/2305.11596/) (2023)

## Built on by (most cited first, 6 of 6)

- [Learning to Reason for Hallucination Span Detection](https://mlsanity.com/paper/2510.02173/) (2025)
- [Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention](https://mlsanity.com/paper/2602.18145/) (2026)
- [CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?](https://mlsanity.com/paper/2607.04029/) (2026)
- [AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detection](https://mlsanity.com/paper/2603.04921/) (2026)
- [Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals](https://mlsanity.com/paper/2605.05025/) (2026)
- [External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing](https://mlsanity.com/paper/2610.02066/) (2026)

## Abstract (arXiv)

This study addresses the problem of hallucinated span detection in the outputs of large language models. It has received less attention than output-level hallucination detection despite its practical importance. Prior work has shown that attentions often exhibit irregular patterns when hallucinations occur. Motivated by these findings, we extract features from the attention matrix that provide complementary views capturing (a) whether certain tokens are influential or ignored, (b) whether attention is biased toward specific subsets, and (c) whether a token is generated referring to a narrow or broad context, in the generation. These features are input to a Transformer-based classifier to conduct sequential labelling to identify hallucinated spans. Experimental results indicate that the proposed method outperforms strong baselines on hallucinated span detection with longer input contexts, such as data-to-text and summarisation tasks.