MODE-RAG: Manifold Outlier Diagnosis and Energy-based Retrieval-Augmented Generation Evaluation
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
also in 1 more
Abstract (arXiv)
While Multimodal Retrieval-Augmented Generation (M-RAG) enhances Large Vision-Language Models, it remains highly susceptible to cross-modal hallucinations, causal fabrications, and sycophancy. Furthermore, existing mitigation pipelines often face an intervention paradox: static rules tend to unnecessarily disrupt accurate generations, whereas leaving the multi-modal reasoning completely unguided allows existing mismatches to cascade into severe logical fabrications. To quantify and mitigate these hallucinations, we propose a Multi-Agent system, MODE-RAG, driven by Variational Free Energy (VFE) and internal attention states to dynamically gate interventions. High-risk queries are routed to five stage-specific agents, integrating Monte Carlo Tree Search (MCTS) for rigorous causal derivation and logit perturbations to penalize sycophancy. Dedicated Correction and Overseer agents ensure formatting stability and perform post-hoc factual verification. To objectively evaluate our approach, we introduce ModeVent, a challenging subset derived from the MultiVent dataset. Extensive experiments indicate that our system effectively reduces hallucination rates and logical fabrication, significantly improving the robustness of M-RAG systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 29 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
13 of 29 references.
References · 13
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
- 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
- 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
- 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
- 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 77
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG
- 2025RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance
- 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
- 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
- 2026Mitigating Hallucination on Hallucination in RAG via Ensemble Voting
- 2026Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and Mitigation
- 2025FVA-RAG: Falsification-Verification Alignment for Mitigating Sycophantic Hallucinations
- 2025The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systems