mlsanityData to 7 Oct 2026

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

arXiv 1706.02275 ↗First posted 7 Jun 2017cs.LG cs.AI cs.NEcited by 1,519

Not on the map. Why?

Cited by 116 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We explore deep reinforcement learning methods for multi-agent domains. We begin by analyzing the difficulty of traditional algorithms in the multi-agent case: Q-learning is challenged by an inherent non-stationarity of the environment, while policy gradient suffers from a variance that increases as the number of agents grows. We then present an adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination. Additionally, we introduce a training regimen utilizing an ensemble of policies for each agent that leads to more robust multi-agent policies. We show the strength of our approach compared to existing methods in cooperative as well as competitive scenarios, where agent populations are able to discover various physical and informational coordination strategies.

Key facts. Cited by 1,519 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 2; 2020: 1; 2021: 235; 2022: 245; 2023: 261; 2024: 228; 2025: 281; 2026: 265.

1,519 papers.

Sort:

By area:

  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  2. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  3. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  4. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  5. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  6. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  7. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  8. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  9. 2021Collaborating with Humans without Human Datacited by 139
  10. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  11. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  12. 2024Empowering Biomedical Discovery with AI Agentscited by 117
38 more
  1. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  2. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  3. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  4. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  5. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  6. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  7. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  8. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 96
  9. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
  10. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  11. 2023MindAgent: Emergent Gaming Interactioncited by 87
  12. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
  13. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  14. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  15. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  16. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 71
  17. 2024Simplifying Deep Temporal Difference Learningcited by 71
  18. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  19. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  20. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  21. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  22. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  23. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  24. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Modelscited by 65
  25. 2021When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?cited by 63
  26. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  27. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  28. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 60
  29. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 60
  30. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  31. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  32. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  33. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  34. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  35. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  36. 2025Embodied AI Agents: Modeling the Worldcited by 57
  37. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  38. 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 54

50 most cited of 1,519 All 1,519 (JSON)