mlsanityData to 14 Oct 2026

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

arXiv 2608.10835 ↗First posted 11 Aug 2026cs.CV cs.LGcited by 0

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input. Effective mitigation requires token-level localization, enabling targeted intervention without discarding the entire response. Existing detectors require expensive full-model fine-tuning, rely on external verifiers that ignore the model's generation process, or reduce internal signals to isolated features and hand-crafted statistics, discarding spatial, sequential, and relational structure. We introduce UniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass. UniProbe constructs a directed graph over image patches, query tokens, and generated tokens, with attention weights encoding their relations. It processes this trace with alternating structure-aware modules: a GNN for relational evidence, a ViT for 2-D visual geometry, and a GRU for response order. Interleaving them allows spatial, relational, and sequential evidence to interact throughout the detector. We further develop a streaming variant for hallucination-aware decoding, which detects and resamples hallucinated tokens during generation, and a self-adaptation strategy aligning the detector with the LVLM's own generations. Across diverse LVLM backbones, UniProbe achieves state-of-the-art token-level and object-hallucination detection. During decoding, it reduces object hallucinations by up to 55% at 1.06× the latency of standard generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 21 of its 32 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

21 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations6 Apr

Other references · 20

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  8. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  9. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 143
  10. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  11. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  12. 2024Where's Waldo: Diffusion Features for Personalized Segmentation and Retrievalcited by 18
8 more
  1. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 17
  2. 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Tokencited by 14
  3. 2024MetaToken: Detecting Hallucination in Image Descriptions by Meta Classificationcited by 13
  4. 2025Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributionscited by 12
  5. 2025TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Interventioncited by 11
  6. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 6
  7. 2025ZINA: Multimodal Fine-grained Hallucination Detection and Editingcited by 5
  8. 2024Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Datacited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs
  2. 2023Detecting and Preventing Hallucinations in Large Vision Language Models
  3. 2025Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding
  4. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
  5. 2023Mitigating Hallucination in Visual Language Models with Visual Supervision
  6. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answering
  7. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base
  8. 2025Beyond Single Models: Mitigating Multimodal Hallucinations via Adaptive Token Ensemble Decoding