mlsanityData to 14 Oct 2026

TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention

arXiv 2503.10602 ↗First posted 13 Mar 2025cs.CV cs.AI cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Object Hallucination (OH) has been acknowledged as one of the major trustworthy challenges in Large Vision-Language Models (LVLMs). Recent advancements in Large Language Models (LLMs) indicate that internal states, such as hidden states, encode the "overall truthfulness" of generated responses. However, it remains under-explored how internal states in LVLMs function and whether they could serve as "per-token" hallucination indicators, which is essential for mitigating OH. In this paper, we first conduct an in-depth exploration of LVLM internal states with OH issues and discover that (1) LVLM internal states are high-specificity per-token indicators of hallucination behaviors. Moreover, (2) different LVLMs encode universal patterns of hallucinations in common latent subspaces, indicating that there exist "generic truthful directions" shared by various LVLMs. Based on these discoveries, we propose Truthful-Guided Pre-Intervention (TruthPrInt) that first learns the truthful direction of LVLM decoding and then applies truthful-guided inference-time intervention during LVLM decoding. We further propose TruthPrInt to enhance both cross-LVLM and cross-data hallucination detection transferability by constructing and aligning hallucination latent subspaces. We evaluate TruthPrInt in extensive experimental settings, including in-domain and out-of-domain scenarios, over popular LVLMs and OH benchmarks. Experimental results indicate that TruthPrInt significantly outperforms state-of-the-art methods. Codes will be available at https://github.com/jinhaoduan/TruthPrInt.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 63 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

55 of 63 references.

Closest ancestors · 21

Same direction, newest first

  1. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection18 Dec
  2. 2024Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models30 Oct
  3. 2024Mitigating Object Hallucination via Concentric Causal Attention21 Oct
  4. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  5. 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMs25 Sep
  6. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs19 Aug
  7. 2024Reference-free Hallucination Detection for Large Vision-Language Models11 Aug
  8. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  9. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  10. 2024Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?18 Jun
  11. 2024CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models4 Jun
  12. 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
9 more
  1. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  2. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  3. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  4. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  5. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  6. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  7. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  8. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  9. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
22 more
  1. 2022Mass-Editing Memory in a Transformercited by 1,001
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  3. 2018Object Hallucination in Image Captioningcited by 669
  4. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  5. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  6. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  7. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  8. 2023Explainability for Large Language Models: A Surveycited by 333
  9. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  10. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  11. 2022Relative representations enable zero-shot latent space communicationcited by 191
  12. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  13. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  14. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  15. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 74
  16. 2024Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Trainingcited by 64
  17. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 55
  18. 2023Unveiling the Pitfalls of Knowledge Editing for Large Language Modelscited by 43
  19. 2023Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Modelscited by 23
  20. 2024PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamicscited by 14
  21. 2024Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Modelscited by 12
  22. 2023LLMMaps -- A Visual Metaphor for Stratified Evaluation of Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineeringcited by 8
  2. 2025On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Modelscited by 5
  3. 2026VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluationcited by 5
  4. 2025Seeing It or Not? Interpretable Vision-aware Latent Steering to Mitigate Object Hallucinationscited by 3
  5. 2025"It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Modelscited by 3
  6. 2025Preemptive Hallucination Reduction: An Input-Level Approach for Multimodal Language Modelcited by 2
  7. 2026IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generationcited by 1
  8. 2026Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigationcited by 1
  9. 2026UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representationscited by 0
  10. 2026Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Modelscited by 0
  11. 2026DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs
  2. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  3. 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
  4. 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token
  5. 2026HIME: Mitigating Object Hallucinations in LVLMs via Hallucination Insensitivity Model Editing
  6. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  7. 2025Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding
  8. 2026What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness