mlsanityData to 7 Oct 2026

Collaborative Memory for Multi-Agent VLM Systems

arXiv 2609.17921 ↗First posted 15 Sep 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Vision-language model (VLM) agents combine specialized perception, tools, and reasoning to address complex visual tasks. In multi-agent settings, different agents inspect different image regions, video frames, or visual representations, so collaboration extends beyond distributed reasoning to distributed perception. This makes shared visual context a central problem in VLM agent collaboration. In this paper, we frame memory hierarchy, cross-agent sharing, and consistency mechanisms around the need to reconcile interpretations and update dependent reasoning. Effective collaboration requires agents to build on contributions from other agents, recover missing visual context, and reconcile differing interpretations as new evidence emerges. Shared visual memory preserves not only images or textual summaries but also the dependencies among observations, agent interpretations, and subsequent reasoning. Together, these design considerations shape how information flows and evolves across VLM agents. The proposed framework provides a foundation for building reliable and resource-efficient agent teams.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 54 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

39 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning27 Jan

Other references · 38

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  9. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  10. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  11. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  12. 2025Memory in the Age of AI Agentscited by 285
26 more
  1. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  2. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
  3. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  4. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  5. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  6. 2025Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Controlcited by 56
  7. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  8. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 50
  9. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 33
  10. 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
  11. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 9
  12. 2026Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understandingcited by 8
  13. 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learningcited by 6
  14. 2026MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillationcited by 6
  15. 2026Self-Evolving Multi-Agent Systems via Decentralized Memorycited by 6
  16. 2026SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memorycited by 5
  17. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaborationcited by 4
  18. 2026Personal Visual Memory from Explicit and Implicit Evidencecited by 2
  19. 2026Beyond Memory Majority: Latent-Source Reasoning for Multi-Agent Memory Arbitrationcited by 2
  20. 2026Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidencecited by 1
  21. 2026MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QAcited by 1
  22. 2026Beyond Retrieval: Analytic Memory for Multimodal Agentscited by 1
  23. 2026GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memorycited by 1
  24. 2026Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curationcited by 1
  25. 2026When History Is Multimodal: Rethinking Context Management for Long-Horizon Agentscited by 1
  26. 2026SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
  2. 2025VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments
  3. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems
  4. 2026Visual Para-Thinker++: A Single-Policy Multi-Agent Framework for Visual Reasoning
  5. 2026Multi-Turn Multi-Agent Dialogue for Collaborative Reconstruction Improves VLM Performance on Spatial Reasoning, But Only Barely
  6. 2025GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning
  7. 2026Diagnosing Failure Modes of Shared-State Collaboration in Resource-Constrained Visual Agents
  8. 2025Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memory