# EAGLE: Enhanced Visual Grounding Minimizes Hallucinations in Instructional Multimodal Models

arXiv: https://arxiv.org/abs/2501.02699 · HTML: https://mlsanity.com/paper/2501.02699/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2025.

## Built on by (most cited first, 8 of 8)

- [SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues](https://mlsanity.com/paper/2603.19092/) (2026)
- [MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessment](https://mlsanity.com/paper/2508.17290/) (2025)
- [Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings](https://mlsanity.com/paper/2511.05017/) (2025)
- [Empowering Multimodal LLMs with External Tools: A Comprehensive Survey](https://mlsanity.com/paper/2508.10955/) (2025)
- [ReLoop: "Seeing Twice and Thinking Backwards" via Closed-loop Training to Mitigate Hallucinations in Multimodal understanding](https://mlsanity.com/paper/2507.04943/) (2025)
- [CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation](https://mlsanity.com/paper/2601.15408/) (2026)
- [MoDA: Modulation Adapter for Fine-Grained Visual Grounding in Instructional MLLMs](https://mlsanity.com/paper/2506.01850/) (2025)
- [DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models](https://mlsanity.com/paper/2607.23944/) (2026)

## Abstract (arXiv)

Large language models and vision transformers have demonstrated impressive zero-shot capabilities, enabling significant transferability in downstream tasks. The fusion of these models has resulted in multi-modal architectures with enhanced instructional capabilities. Despite incorporating vast image and language pre-training, these multi-modal architectures often generate responses that deviate from the ground truth in the image data. These failure cases are known as hallucinations. Current methods for mitigating hallucinations generally focus on regularizing the language component, improving the fusion module, or ensembling multiple visual encoders to improve visual representation. In this paper, we address the hallucination issue by directly enhancing the capabilities of the visual component. Our approach, named EAGLE, is fully agnostic to the LLM or fusion module and works as a post-pretraining approach that improves the grounding and language alignment of the visual encoder. We show that a straightforward reformulation of the original contrastive pre-training task results in an improved visual encoder that can be incorporated into the instructional multi-modal architecture without additional instructional training. As a result, EAGLE achieves a significant reduction in hallucinations across multiple challenging benchmarks and tasks.