mlsanityData to 7 Oct 2026

Latent Collaboration in Multi-Agent Systems

arXiv 2511.20639 ↗First posted 25 Nov 2025cs.CL cs.AI cs.LGcited by 67

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings instead of text. Then, a shared latent working memory preserves and transfers each agent's internal representations and latent thoughts, ensuring lossless information exchange without re-encoding. We provide detailed theoretical analyses showing that LatentMAS achieves higher expressiveness and lossless information preservation with lower overall complexity than standard text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS outperforms advanced single agents and text-based MAS baselines, achieving up to 14.6% higher accuracy, reducing output token usage by 70.8%-83.7%, and providing 4×-4.3× faster end-to-end inference. Code and data are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 63 of its 77 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

63 of 77 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Thought Communication in Multiagent Collaboration23 Oct
  2. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation29 May
  3. 2025ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks4 Mar
  4. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning7 Feb
  5. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  6. 2024Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions2 Feb

Other references · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  10. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  11. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  12. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
38 more
  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  5. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  6. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  8. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  9. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  10. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  11. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  12. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  13. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  14. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 183
  15. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  16. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  17. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  18. 2025Latent Visual Reasoningcited by 99
  19. 2024Language Agents as Optimizable Graphscited by 81
  20. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 73
  21. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
  22. 2025A Survey on Latent Reasoningcited by 71
  23. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  24. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 58
  25. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 54
  26. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  27. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  28. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  29. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscited by 35
  30. 2024ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoningcited by 31
  31. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 31
  32. 2024Deliberation in Latent Space via Differentiable Cache Augmentationcited by 25
  33. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  34. 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
  35. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systemscited by 17
  36. 2025KVComm: Enabling Efficient LLM Communication through Selective KV Sharingcited by 16
  37. 2025Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Saycited by 12
  38. 2025A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Gamescited by 9

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 65.

67 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  2. 2026Code as Agent Harnesscited by 40
  3. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulatorscited by 34
  4. 2026Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baselinecited by 18
  5. 2026Multi-Agent Teams Hold Experts Backcited by 16
  6. 2026Learning to Continually Learn via Meta-learning Agentic Memory Designscited by 16
  7. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  8. 2026Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systemscited by 11
  9. 2026SAM: State-Adaptive Memory for Long-Horizon Reasoning Agentcited by 10
  10. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 9
  11. 2026Influence-Preserving Proxies for Gradient-Based Data Selection in LLM Fine-tuningcited by 9
  12. 2025An Information Theoretic Perspective on Agentic System Designcited by 8
38 more
  1. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 8
  2. 2026Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systemscited by 6
  3. 2026TodyComm: Task-Oriented Dynamic Communication for Multi-Round LLM-based Multi-Agent Systemcited by 5
  4. 2026DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verificationcited by 5
  5. 2026Token Economics for LLM Agents: A Dual-View Study from Computing and Economicscited by 5
  6. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  7. 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systemscited by 5
  8. 2026StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systemscited by 5
  9. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaborationcited by 4
  10. 2026Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Tracescited by 4
  11. 2026KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scalingcited by 4
  12. 2026Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensuscited by 3
  13. 2026MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systemscited by 3
  14. 2026When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMscited by 3
  15. 2026When Numbers Start Talking: Implicit Numerical Coordination Among LLM-Based Agentscited by 2
  16. 2026What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysiscited by 2
  17. 2026When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaborationcited by 2
  18. 2026XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communicationcited by 2
  19. 2026Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoningcited by 1
  20. 2026Graph homophily booster: Reimagining the role of discrete features in heterophilic graph learningcited by 1
  21. 2026ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuningcited by 1
  22. 2026LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphscited by 1
  23. 2026MeloTune: On-Device Arousal Learning and Peer-to-Peer Mood Coupling for Proactive Music Curationcited by 1
  24. 2026QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMscited by 1
  25. 2026A Token/KV-Cache Communication Media Selection and Resource Allocation Strategy for Multi-Agent Collaborationcited by 1
  26. 2026Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMscited by 1
  27. 2026DH-VLM: Dual-Horizon Cooperative Latent Reasoning for Autonomous Drivingcited by 1
  28. 2026Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agentscited by 1
  29. 2026Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Sciencecited by 0
  30. 2026Position: Introspective Experience from Conversational Environments as a Path to Better Learningcited by 0
  31. 2026When Less is Enough: Efficient Inference via Collaborative Reasoningcited by 0
  32. 2026LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verificationcited by 0
  33. 2026Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weightscited by 0
  34. 2026LACO: Adaptive Latent Communication for Collaborative Drivingcited by 0
  35. 2026Closing the Loop on Latent Reasoning via Test-Time Reconstructioncited by 0
  36. 2026Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflowscited by 0
  37. 2026Tandem Reinforcement Learning with Verifiable Rewardscited by 0
  38. 2026How Far Do Simple Transformations Translate Across Text Embedding Models?cited by 0

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systems
  2. 2025Enabling Agents to Communicate Entirely in Latent Space
  3. 2026LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration
  4. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systems
  5. 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
  6. 2026Can Small Agents Collaborate to Beat a Single Large Language Model?
  7. 2026Recursive Multi-Agent Systems
  8. 2026Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning