CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies
Not on the map. Why?
Abstract (arXiv)
Multi-agent workflows require task-relevant information to be shared across agents, while irrelevant, stale, unverified, or incompatible information must remain isolated. We call this task-conditioned scope of information a collaborative memory boundary. Workflow topology determines which intermediate artifacts are applicable to which downstream workers and when they cease to be valid, thereby providing a structural stress dimension for sharing and isolation. Existing memory benchmarks primarily evaluate retention and retrieval, whereas multi-agent benchmarks emphasize coordination and end-to-end completion, leaving topology-conditioned memory boundaries largely unmeasured. We introduce CoMemBench, an execution-grounded benchmark for collaborative memory sharing and isolation across multi-agent workflow topologies. It constructs 800 composite workflows across four domains from source-grounded dependency graphs, with node-local specifications, verifiable artifact handoffs, native evaluators, and matched isolation challenges. CoMemBench measures workflow completion, verified node progress, required-handoff reliability, isolation robustness, and token cost. Experiments reveal a sharing-isolation trade-off: broader context improves information availability but can weaken isolation, while system rankings shift across topologies and artifact violations.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
34 of 41 references.
References · 34
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
22 more
- 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
- 2024Automated Design of Agentic Systemscited by 407
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 178
- 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 144
- 2023MindAgent: Emergent Gaming Interactioncited by 88
- 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 85
- 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Modelscited by 68
- 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaborationcited by 51
- 2024miniCTX: Neural Theorem Proving with (Long-)Contextscited by 37
- 2025REALTALK: A 21-Day Real-World Dataset for Long-Term Conversationcited by 35
- 2026MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarkscited by 32
- 2026PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agentscited by 32
- 2024SMoA: Improving Multi-agent Large Language Models with Sparse Mixture-of-Agentscited by 30
- 2025StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turnscited by 20
- 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systemscited by 20
- 2026GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversationscited by 17
- 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 13
- 2026PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraintscited by 7
- 2025OMAC: A Holistic Optimization Framework for LLM-Based Multi-Agent Collaborationcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Topological Coherence for Self-evolving Multi-agent Systems29 Sep 2026