# InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration

arXiv: https://arxiv.org/abs/2512.02981 · HTML: https://mlsanity.com/paper/2512.02981/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)
- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 25 of 25)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning](https://mlsanity.com/paper/2306.14565/) (2023)
- [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://mlsanity.com/paper/2312.00849/) (2023)
- [In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer](https://mlsanity.com/paper/2504.20690/) (2025)
- [Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding](https://mlsanity.com/paper/2403.18715/) (2024)
- [HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding](https://mlsanity.com/paper/2403.00425/) (2024)
- [Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models](https://mlsanity.com/paper/2408.02032/) (2024)
- [Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback](https://mlsanity.com/paper/2404.14233/) (2024)
- [LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling](https://mlsanity.com/paper/2511.20785/) (2025)
- [PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training](https://mlsanity.com/paper/2503.06486/) (2025)
- [OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe](https://mlsanity.com/paper/2511.16334/) (2025)
- [Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling](https://mlsanity.com/paper/2504.13169/) (2025)
- [Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification](https://mlsanity.com/paper/2407.02352/) (2024)
- [WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generation](https://mlsanity.com/paper/2503.19065/) (2025)

## Built on by (most cited first, 5 of 5)

- [Evaluating Long-Horizon Memory for Multi-Party Collaborative Dialogues](https://mlsanity.com/paper/2602.01313/) (2026)
- [ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning](https://mlsanity.com/paper/2605.20342/) (2026)
- [SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaboration](https://mlsanity.com/paper/2604.05079/) (2026)
- [MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents](https://mlsanity.com/paper/2603.05697/) (2026)
- [Decoding by Perturbation: Mitigating MLLM Hallucinations via Dynamic Textual Perturbation](https://mlsanity.com/paper/2604.12424/) (2026)

## Abstract (arXiv)

Hallucination remains a critical challenge in large language models (LLMs), hindering the development of reliable multimodal LLMs (MLLMs). Existing solutions often rely on human intervention or underutilize the agent's ability to autonomously mitigate hallucination. To address these limitations, we draw inspiration from how humans make reliable decisions in the real world. They begin with introspective reasoning to reduce uncertainty and form an initial judgment, then rely on external verification from diverse perspectives to reach a final decision. Motivated by this cognitive paradigm, we propose InEx, a training-free, multi-agent framework designed to autonomously mitigate hallucination. InEx introduces internal introspective reasoning, guided by entropy-based uncertainty estimation, to improve the reliability of the decision agent's reasoning process. The agent first generates a response, which is then iteratively verified and refined through external cross-modal multi-agent collaboration with the editing agent and self-reflection agents, further enhancing reliability and mitigating hallucination. Extensive experiments show that InEx consistently outperforms existing methods, achieving 4%-27% gains on general and hallucination benchmarks, and demonstrating strong robustness.