# Decentralized Multi-Agent Systems with Shared Context arXiv: https://arxiv.org/abs/2606.10662 ยท HTML: https://mlsanity.com/paper/2606.10662/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 38 of its 54 references that are in the map. - First posted on arXiv 9 Jun 2026. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 30 of 38) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022) - [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023) - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023) - [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023) - [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023) - [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023) - [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024) - [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023) - [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023) - [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026) - [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023) - [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024) - [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023) - [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023) - [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025) - [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026) - [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025) - [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026) - [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023) - [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024) - [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023) - [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025) - [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024) - [Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation](https://mlsanity.com/paper/2307.15337/) (2023) - [Recursive Language Models](https://mlsanity.com/paper/2512.24601/) (2025) - [MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models](https://mlsanity.com/paper/2505.22101/) (2025) - [ProgramBench: Can Language Models Rebuild Programs From Scratch?](https://mlsanity.com/paper/2605.03546/) (2026) - [ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration](https://mlsanity.com/paper/2511.21689/) (2025) - [Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities](https://mlsanity.com/paper/2511.02817/) (2025) ## Built on by (most cited first, 10 of 10) - [Prove2Me: An Open Collaborative Platform for Scaling Math Formalization](https://mlsanity.com/paper/2608.28433/) (2026) - [EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?](https://mlsanity.com/paper/2609.04280/) (2026) - [Architectural Implications of Agentic AI Workflows](https://mlsanity.com/paper/2608.04458/) (2026) - [Agensh: Scaling Organizational Intelligence to 1,024 Agents](https://mlsanity.com/paper/2609.26781/) (2026) - [Spine-Branch Coordination for Multi-agent Computer Use](https://mlsanity.com/paper/2608.22077/) (2026) - [Proxifield: Decentralized Multi-Agent Communication through Semantic Proximity](https://mlsanity.com/paper/2609.20889/) (2026) - [Sentence Specificity Scores for Collaborative Technical Documentation: A Domain-Transfer Study](https://mlsanity.com/paper/2610.01046/) (2026) - [Scaling Multi-Agent Intelligence with Substrate-Centric Coordination](https://mlsanity.com/paper/2610.16120/) (2026) - [Degrees of Freedom in Multi-Agent Coordination: Between Centralized and Decentralized Regimes](https://mlsanity.com/paper/2610.17751/) (2026) - [Scaling Discovery Requires Forgetting](https://mlsanity.com/paper/2610.20563/) (2026) ## Abstract (arXiv) Multi-agent systems (MAS) can scale large language model agents on long-horizon tasks by running them in parallel, yet existing designs waste much of this parallelism in bubbles: agent time spent waiting on others or redoing a peer's work. These bubbles stem from how agents communicate. Independent agents share nothing and rediscover what their peers have already found; peer-communicating agents wait at synchronous rounds; and under centralized orchestration, the main agent blocks on its sub-agents while progress is relayed. We propose Decentralized Language Models (DeLM), a MAS framework on top of existing agent harnesses that squeezes out these bubbles by replacing the main agent with a shared context and a task queue. Agents asynchronously claim tasks, publish findings as soon as they are available, and build on or correct one another's progress, with every peer's status visible to all. On long-horizon tasks from Terminal-Bench 4.0 and DeepSWE v1.1, and on SWE-bench Verified, DeLM is both more accurate and faster than Codex, Claude Code, their native subagents, and AOrchestra in every setting, improving accuracy by up to 17.5 points over the strongest baseline and running up to 2.49x faster than the harness it builds on. On ProgramBench, where agents rebuild programs from scratch, DeLM makes faster progress than Claude Code and finishes a 120-minute budget up to 19.9 points higher in test pass rate. The code is available on our project website at https://yuzhenmao.github.io/DeLM/.