# TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection

arXiv: https://arxiv.org/abs/2504.04099 · HTML: https://mlsanity.com/paper/2504.04099/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 5 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 28 of 28)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation](https://mlsanity.com/paper/2311.07397/) (2023)
- [Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization](https://mlsanity.com/paper/2311.16839/) (2023)
- [LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents](https://mlsanity.com/paper/2311.05437/) (2023)
- [IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding](https://mlsanity.com/paper/2402.18476/) (2024)
- [Cross-modal Information Flow in Multimodal Large Language Models](https://mlsanity.com/paper/2411.18620/) (2024)
- [Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models](https://mlsanity.com/paper/2403.12966/) (2024)
- [Reducing Hallucinations in Vision-Language Models via Latent Space Steering](https://mlsanity.com/paper/2410.15778/) (2024)
- [Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs](https://mlsanity.com/paper/2411.09968/) (2024)
- [Mitigating Object Hallucination via Concentric Causal Attention](https://mlsanity.com/paper/2410.15926/) (2024)
- [Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance](https://mlsanity.com/paper/2411.14279/) (2024)
- [Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models](https://mlsanity.com/paper/2407.00569/) (2024)
- [DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer](https://mlsanity.com/paper/2407.15130/) (2024)
- [Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment](https://mlsanity.com/paper/2405.17871/) (2024)
- [DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models](https://mlsanity.com/paper/2402.14767/) (2024)

## Built on by (most cited first, 7 of 7)

- [A Survey of Multimodal Hallucination Evaluation and Detection](https://mlsanity.com/paper/2507.19024/) (2025)
- [Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention](https://mlsanity.com/paper/2511.20032/) (2025)
- [Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction](https://mlsanity.com/paper/2512.18813/) (2025)
- [On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2510.09008/) (2025)
- [Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation](https://mlsanity.com/paper/2510.22067/) (2025)
- [Cross-Layer Vision Smoothing: Enhancing Visual Understanding via Sustained Focus on Key Objects in Large Vision-Language Models](https://mlsanity.com/paper/2509.12897/) (2025)
- [SAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplification](https://mlsanity.com/paper/2602.13600/) (2026)

## Abstract (arXiv)

Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead (~4% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency.