# A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment

arXiv: https://arxiv.org/abs/2312.01058 · HTML: https://mlsanity.com/paper/2312.01058/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2023.

## Built on by (most cited first, 30 of 39)

- [Multi-agent Embodied AI: Advances and Future Directions](https://mlsanity.com/paper/2505.05108/) (2025)
- [Agentic LLMs in the Supply Chain: Towards Autonomous Multi-Agent Consensus-Seeking](https://mlsanity.com/paper/2411.10184/) (2024)
- [N-Agent Ad Hoc Teamwork](https://mlsanity.com/paper/2404.10740/) (2024)
- [Context-aware Communication for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2312.15600/) (2023)
- [MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2506.14990/) (2025)
- [Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation](https://mlsanity.com/paper/2406.09068/) (2024)
- [Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Exploration](https://mlsanity.com/paper/2410.02511/) (2024)
- [C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.06205/) (2025)
- [Q-Adapter: Customizing Pre-trained LLMs to New Preferences with Forgetting Mitigation](https://mlsanity.com/paper/2407.03856/) (2024)
- [Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.08540/) (2024)
- [Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2405.18110/) (2024)
- [Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.12001/) (2024)
- [Triple-BERT: Do We Really Need MARL for Order Dispatch on Ride-Sharing Platforms?](https://mlsanity.com/paper/2510.03257/) (2025)
- [Efficient Reinforcement Learning via Decoupling Exploration and Utilization](https://mlsanity.com/paper/2312.15965/) (2023)
- [LLM-Mediated Guidance of MARL Systems](https://mlsanity.com/paper/2503.13553/) (2025)
- [Sequential Multi-Agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2510.23535/) (2025)
- [Contrastive learning-based agent modeling for deep reinforcement learning](https://mlsanity.com/paper/2401.00132/) (2023)
- [Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2507.23604/) (2025)
- [Unifying Agent Interaction and World Information for Multi-agent Coordination](https://mlsanity.com/paper/2509.25550/) (2025)
- [MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.04935/) (2025)
- [Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensus](https://mlsanity.com/paper/2604.13472/) (2026)
- [Efficient Test-Time Optimization for Multi-Agent Proof Autoformalization](https://mlsanity.com/paper/2607.11307/) (2026)
- [HOLA-Drone: Hypergraphic Open-ended Learning for Zero-Shot Multi-Drone Cooperative Pursuit](https://mlsanity.com/paper/2409.08767/) (2024)
- [A Multi-Agent Reinforcement Learning Testbed for Cognitive Radio Applications](https://mlsanity.com/paper/2410.21521/) (2024)
- [Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review](https://mlsanity.com/paper/2507.10142/) (2025)
- [SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework](https://mlsanity.com/paper/2509.18167/) (2025)
- [Autonomous Aerial Manipulation via Contextual Contrastive Meta Reinforcement Learning](https://mlsanity.com/paper/2606.08533/) (2026)
- [JaxAHT: A JAX-Based Library for Ad Hoc Teamwork](https://mlsanity.com/paper/2609.13716/) (2026)
- [Deep Learning Agents Trained For Avoidance Behave Like Hawks And Doves](https://mlsanity.com/paper/2503.11452/) (2025)
- [Distributed Area Coverage with High Altitude Balloons Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.03823/) (2025)

## Abstract (arXiv)

Multi-agent Reinforcement Learning (MARL) has gained wide attention in recent years and has made progress in various fields. Specifically, cooperative MARL focuses on training a team of agents to cooperatively achieve tasks that are difficult for a single agent to handle. It has shown great potential in applications such as path planning, autonomous driving, active voltage control, and dynamic algorithm configuration. One of the research focuses in the field of cooperative MARL is how to improve the coordination efficiency of the system, while research work has mainly been conducted in simple, static, and closed environment settings. To promote the application of artificial intelligence in real-world, some research has begun to explore multi-agent coordination in open environments. These works have made progress in exploring and researching the environments where important factors might change. However, the mainstream work still lacks a comprehensive review of the research direction. In this paper, starting from the concept of reinforcement learning, we subsequently introduce multi-agent systems (MAS), cooperative MARL, typical methods, and test environments. Then, we summarize the research work of cooperative MARL from closed to open environments, extract multiple research directions, and introduce typical works. Finally, we summarize the strengths and weaknesses of the current research, and look forward to the future development direction and research problems in cooperative MARL in open environments.