mlsanityData to 14 Oct 2026

Hallucinated Span Detection with Multi-View Attention Features

arXiv 2504.04335 ↗First posted 6 Apr 2025cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This study addresses the problem of hallucinated span detection in the outputs of large language models. It has received less attention than output-level hallucination detection despite its practical importance. Prior work has shown that attentions often exhibit irregular patterns when hallucinations occur. Motivated by these findings, we extract features from the attention matrix that provide complementary views capturing (a) whether certain tokens are influential or ignored, (b) whether attention is biased toward specific subsets, and (c) whether a token is generated referring to a narrow or broad context, in the generation. These features are input to a Transformer-based classifier to conduct sequential labelling to identify hallucinated spans. Experimental results indicate that the proposed method outperforms strong baselines on hallucinated span detection with longer input contexts, such as data-to-text and summarisation tasks.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

14 of 32 references.

References · 14

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2019Revealing the Dark Secrets of BERTcited by 301
  6. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  7. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  8. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  9. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  10. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 88
  11. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  12. 2019TENER: Adapting Transformer Encoder for Named Entity Recognitioncited by 69
2 more
  1. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  2. 2023Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlationcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Learning to Reason for Hallucination Span Detectioncited by 5
  2. 2026Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attentioncited by 2
  3. 2026CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?cited by 2
  4. 2026AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detectioncited by 0
  5. 2026Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signalscited by 0
  6. 2026External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probingcited by 0