mlsanityData to 14 Oct 2026

Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flow

arXiv 2509.21789 ↗First posted 26 Sep 2025cs.MA cs.CVcited by 13

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Multi-Agent System (MAS) powered by Visual Language Models (VLMs) enables challenging tasks but suffers from a novel failure term, multi-agent visual hallucination snowballing, where hallucinations are seeded in a single agent and amplified by following ones due to the over-reliance on textual flow to relay visual information. Through turn-, layer-, and token-wise attention analyses, we provide detailed insights into the essence of hallucination snowballing regarding the reduction of visual attention allocation. It leads us to identify a subset of vision tokens with a unimodal attention peak in middle layers that best preserve visual evidence but gradually diminish in deeper agent turns, resulting in the visual hallucination snowballing in MAS. Thus, we propose ViF, a lightweight, plug-and-play mitigation paradigm that relays inter-agent messages with Visual Flow powered by the selected visual relay tokens and applies attention reallocation to amplify this pattern. The experiment results demonstrate that our method markedly reduces hallucination snowballing, consistently improving the performance across eight benchmarks based on four common MAS structures and ten base models. The source code is publicly available at: https://github.com/YU-deep/ViF.git.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 57 of its 68 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

57 of 68 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding22 May
  2. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models17 Mar
  3. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering5 Feb
  4. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  5. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  6. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  7. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  8. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  9. 2024Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models30 Jun
  10. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  11. 2024Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination21 Mar
  12. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
5 more
  1. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  2. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
  3. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  4. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
  5. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct

Other references · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
28 more
  1. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  4. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  5. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  6. 2023A Survey on Multimodal Large Language Modelscited by 909
  7. 2023Vision Transformers Need Registerscited by 871
  8. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  9. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  11. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  12. 2023How Language Model Hallucinations Can Snowballcited by 311
  13. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  14. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  15. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  16. 2025Multi-agent Architecture Search via Agentic Supernetcited by 179
  17. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  18. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  19. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 155
  20. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
  21. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 119
  22. 2025Ovis2.5 Technical Reportcited by 108
  23. 2024Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimizationcited by 84
  24. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  25. 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 67
  26. 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Modelscited by 62
  27. 2022Avoiding spurious correlations via logit correctioncited by 32
  28. 2025Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encodingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 8.

13 papers.

Sort:

By area:

  1. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  2. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  3. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
  4. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attentioncited by 11
  5. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 10
  6. 2026SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topologycited by 9
  7. 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 8
  8. 2025LungNoduleAgent: A Collaborative Multi-Agent System for Precision Diagnosis of Lung Nodulescited by 4
  9. 2026PATRA: Pattern-Aware Alignment and Balanced Reasoning for Time Series Question Answeringcited by 4
  10. 2026Adaptive Debiasing Tsallis Entropy for Test-Time Adaptationcited by 3
  11. 2026Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptationcited by 1
  12. 2026SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMscited by 1
1 more
  1. 2026Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
  2. 2024Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate
  3. 2026VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck
  4. 2026Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
  5. 2026Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
  6. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
  7. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens
  8. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling