mlsanityData to 7 Oct 2026

Distributed Reinforcement Learning for Robot Teams: A Review

arXiv 2204.03516 ↗First posted 7 Apr 2022cs.RO cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Purpose of review: Recent advances in sensing, actuation, and computation have opened the door to multi-robot systems consisting of hundreds/thousands of robots, with promising applications to automated manufacturing, disaster relief, harvesting, last-mile delivery, port/airport operations, or search and rescue. The community has leveraged model-free multi-agent reinforcement learning (MARL) to devise efficient, scalable controllers for multi-robot systems (MRS). This review aims to provide an analysis of the state-of-the-art in distributed MARL for multi-robot cooperation. Recent findings: Decentralized MRS face fundamental challenges, such as non-stationarity and partial observability. Building upon the "centralized training, decentralized execution" paradigm, recent MARL approaches include independent learning, centralized critic, value decomposition, and communication learning approaches. Cooperative behaviors are demonstrated through AI benchmarks and fundamental real-world robotic capabilities such as multi-robot motion/path planning. Summary: This survey reports the challenges surrounding decentralized model-free MARL for multi-robot cooperation and existing classes of approaches. We present benchmarks and robotic applications along with a discussion on current open avenues for research.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 74 of its 124 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

74 of 124 references.

References · 74

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  7. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  11. 2019The StarCraft Multi-Agent Challengecited by 589
  12. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
38 more
  1. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  2. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  3. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  4. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  5. 2016Learning Multiagent Communication with Backpropagationcited by 355
  6. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  7. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  8. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  9. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  10. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  11. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  12. 2018An Introduction to Deep Reinforcement Learningcited by 201
  13. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  14. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  15. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
  16. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  17. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  18. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 159
  19. 2018Graph Convolutional Reinforcement Learningcited by 148
  20. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  21. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  22. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 122
  23. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  24. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
  25. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 90
  26. 2018PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learningcited by 89
  27. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 89
  28. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  29. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 81
  30. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 68
  31. 2017MINOS: Multimodal Indoor Simulator for Navigation in Complex Environmentscited by 66
  32. 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
  33. 2018Emergent Communication through Negotiationcited by 62
  34. 2020PRIMAL2: Pathfinding via Reinforcement and Imitation Multi-Agent Learning -- Lifelongcited by 61
  35. 2019Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcited by 46
  36. 2019Assistive Gym: A Physics Simulation Framework for Assistive Roboticscited by 44
  37. 2019Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agentscited by 42
  38. 2019Learning Transferable Cooperative Behavior in Multi-Agent Teamscited by 40

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 5.

11 papers.

  1. 2024LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Plannercited by 31
  2. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  3. 2024Self-organized free-flight arrival for urban air mobilitycited by 2
  4. 2026Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learningcited by 2
  5. 2025Nonparametric Bellman Mappings for Value Iteration in Distributed Reinforcement Learningcited by 1
  6. 2025Emergence of Roles in Robotic Teams with Model Sharing and Limited Communicationcited by 1
  7. 2026Safe Heterogeneous Multi-Agent RL with Communication Regularization for Coordinated Target Acquisitioncited by 1
  8. 2024Achieving Collective Welfare in Multi-Agent Reinforcement Learning via Suggestion Sharingcited by 0
  9. 2025PACT: Phenotype-Aware Contrastive Team Representation for Multi-Phenotype Grouped Ad Hoc Teamworkcited by 0
  10. 2026SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learningcited by 0
  11. 2026FedGuide: Diffusion Prior Alignment and Value Baseline Guidance for Heterogeneous Federated Reinforcement Learningcited by 0