# A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives

arXiv: https://arxiv.org/abs/2503.13415 · HTML: https://mlsanity.com/paper/2503.13415/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 99 of its 341 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 30 of 99)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://mlsanity.com/paper/2305.17144/) (2023)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2109.11251/) (2021)
- [FACMAC: Factored Multi-Agent Centralised Policy Gradients](https://mlsanity.com/paper/2003.06709/) (2020)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms](https://mlsanity.com/paper/2406.14228/) (2024)
- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games](https://mlsanity.com/paper/1703.10069/) (2017)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities](https://mlsanity.com/paper/2405.10825/) (2024)
- [MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning](https://mlsanity.com/paper/2401.10727/) (2024)

## Built on by (most cited first, 27 of 27)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Internet of Agents: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2505.07176/) (2025)
- [One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence](https://mlsanity.com/paper/2602.03109/) (2026)
- [One Step is Enough: Multi-Agent Reinforcement Learning based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms](https://mlsanity.com/paper/2507.15351/) (2025)
- [Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition](https://mlsanity.com/paper/2606.00959/) (2026)
- [Triple-BERT: Do We Really Need MARL for Order Dispatch on Ride-Sharing Platforms?](https://mlsanity.com/paper/2510.03257/) (2025)
- [Agent-as-a-Service based on Agent Network](https://mlsanity.com/paper/2505.08446/) (2025)
- [Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordination](https://mlsanity.com/paper/2508.14635/) (2025)
- [Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization](https://mlsanity.com/paper/2603.12933/) (2026)
- [Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2507.23604/) (2025)
- [DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration](https://mlsanity.com/paper/2511.04646/) (2025)
- [Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensus](https://mlsanity.com/paper/2604.13472/) (2026)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Overcoming Environmental Meta-Stationarity in MARL via Adaptive Curriculum and Counterfactual Group Advantage](https://mlsanity.com/paper/2506.07548/) (2025)
- [What Would an LLM Do? Evaluating Large Language Models for Policymaking to Alleviate Homelessness](https://mlsanity.com/paper/2509.03827/) (2025)
- [Can Agents Judge Systematic Reviews Like Humans? Evaluating SLRs with LLM-based Multi-Agent System](https://mlsanity.com/paper/2509.17240/) (2025)
- [MARO: Learning Stronger Reasoning from Social Interaction](https://mlsanity.com/paper/2601.12323/) (2026)
- [Self-Evolving Multi-Agent Network for Industrial IoT Predictive Maintenance](https://mlsanity.com/paper/2602.16738/) (2026)
- [Conflict-Resilient Multi-Agent Reasoning via Signed Graph Modeling](https://mlsanity.com/paper/2605.19418/) (2026)
- [ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspective](https://mlsanity.com/paper/2509.21134/) (2025)
- [A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation](https://mlsanity.com/paper/2601.11792/) (2026)
- [DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoning](https://mlsanity.com/paper/2602.08586/) (2026)
- [Story2Proposal: A Scaffold for Structured Scientific Paper Writing](https://mlsanity.com/paper/2603.27065/) (2026)
- [Wireless Communication Enhanced Value Decomposition for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2604.08728/) (2026)
- [Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures](https://mlsanity.com/paper/2604.18133/) (2026)
- [Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimization](https://mlsanity.com/paper/2607.17924/) (2026)
- [RideSkill: A Hierarchical Algorithm for Generalized Ride Sharing with LLM-Driven Automatic Evolution](https://mlsanity.com/paper/2609.02250/) (2026)

## Abstract (arXiv)

With the rapid development of artificial intelligence, intelligent decision-making techniques have gradually surpassed human levels in various human-machine competitions, especially in complex multi-agent cooperative task scenarios. Multi-agent cooperative decision-making involves multiple agents working together to complete established tasks and achieve specific objectives. These techniques are widely applicable in real-world scenarios such as autonomous driving, drone navigation, disaster rescue, and simulated military confrontations. This paper begins with a comprehensive survey of the leading simulation environments and platforms used for multi-agent cooperative decision-making. Specifically, we provide an in-depth analysis for these simulation environments from various perspectives, including task formats, reward allocation, and the underlying technologies employed. Subsequently, we provide a comprehensive overview of the mainstream intelligent decision-making approaches, algorithms and models for multi-agent systems (MAS). Theseapproaches can be broadly categorized into five types: rule-based (primarily fuzzy logic), game theory-based, evolutionary algorithms-based, deep multi-agent reinforcement learning (MARL)-based, and large language models(LLMs)reasoning-based. Given the significant advantages of MARL andLLMs-baseddecision-making methods over the traditional rule, game theory, and evolutionary algorithms, this paper focuses on these multi-agent methods utilizing MARL and LLMs-based techniques. We provide an in-depth discussion of these approaches, highlighting their methodology taxonomies, advantages, and drawbacks. Further, several prominent research directions in the future and potential challenges of multi-agent cooperative decision-making are also detailed.