TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead (~4% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 35 references that are in the map. First posted on arXiv 5 Apr 2025.
Builds on
28 of 35 references.
Closest ancestors · 12
Same direction, newest first
- 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs15 Nov
- 2024Mitigating Object Hallucination via Concentric Causal Attention21 Oct
- 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering21 Oct
- 2024DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer21 Jul
- 2024Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models30 Jun
- 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
- 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
- 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
- 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
- 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
- 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
Other references · 16
- 2023Visual Instruction Tuningcited by 7,792
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
- 2018Object Hallucination in Image Captioningcited by 661
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
- 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Modelscited by 65
4 more
- 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 65
- 2024Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidancecited by 32
- 2024Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignmentcited by 19
- 2024DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Modelscited by 14
Built on by
Citing papers per year: 2025: 6; 2026: 1.
7 papers.
- 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
- 2025Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attentioncited by 6
- 2025Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correctioncited by 6
- 2025On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Modelscited by 4
- 2025Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigationcited by 3
- 2025Cross-Layer Vision Smoothing: Enhancing Visual Understanding via Sustained Focus on Key Objects in Large Vision-Language Modelscited by 2
- 2026SAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplificationcited by 2
- 2026SAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplification14 Feb 2026
- 2025Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction21 Dec 2025
- 2025Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention25 Nov 2025
- 2025Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation24 Oct 2025
- 2025On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models10 Oct 2025
- 2025Cross-Layer Vision Smoothing: Enhancing Visual Understanding via Sustained Focus on Key Objects in Large Vision-Language Models16 Sep 2025
- 2025A Survey of Multimodal Hallucination Evaluation and Detection25 Jul 2025
Similar, not cited
Close in wording, no citation either way
- 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
- 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
- 2026See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
- 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
- 2026Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models
- 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs
- 2025Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
- 2026Attention to details, logits to truth: visual-aware attention and logits enhancement to mitigate hallucinations in LVLMs