mlsanityData to 7 Oct 2026

Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?

arXiv 2011.09533 ↗First posted 18 Nov 2020cs.AIcited by 281

Not on the map. Why?

Cited by 26 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Most recently developed approaches to cooperative multi-agent reinforcement learning in the centralized training with decentralized execution setting involve estimating a centralized, joint value function. In this paper, we demonstrate that, despite its various theoretical shortcomings, Independent PPO (IPPO), a form of independent learning in which each agent simply estimates its local value function, can perform just as well as or better than state-of-the-art joint learning approaches on popular multi-agent benchmark suite SMAC with little hyperparameter tuning. We also compare IPPO to several variants; the results suggest that IPPO's strong performance may be due to its robustness to some forms of environment non-stationarity.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 28; 2023: 41; 2024: 54; 2025: 85; 2026: 59.

281 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  3. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  5. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  6. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  7. 2024Simplifying Deep Temporal Difference Learningcited by 71
  8. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 71
  9. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  10. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  11. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  12. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
38 more
  1. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  2. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
  3. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  4. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
  5. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  6. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  7. 2021Multi-Agent Constrained Policy Optimisationcited by 32
  8. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  9. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  10. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  11. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  12. 2023Breaking the Curse of Multiagents in a Large State Space: RL in Markov Games with Independent Linear Function Approximationcited by 26
  13. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  14. 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 24
  15. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  16. 2021Model-Based Opponent Modelingcited by 21
  17. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  18. 2024InvAgent: A Large Language Model based Multi-Agent System for Inventory Management in Supply Chainscited by 20
  19. 2021Collaborative Visual Navigationcited by 18
  20. 2022Generalization in Cooperative Multi-Agent Systemscited by 18
  21. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 18
  22. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  23. 2022Policy Optimization for Markov Games: Unified Framework and Faster Convergencecited by 16
  24. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 16
  25. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defensecited by 16
  26. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  27. 2025OvercookedV2: Rethinking Overcooked for Zero-Shot Coordinationcited by 14
  28. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordinationcited by 14
  29. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learningcited by 13
  30. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  31. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  32. 2023Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learningcited by 12
  33. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 12
  34. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 12
  35. 2022Trust Region Bounds for Decentralized PPO Under Non-stationaritycited by 11
  36. 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Qualitycited by 11
  37. 2022Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learningcited by 10
  38. 2023An Analysis of Multi-Agent Reinforcement Learning for Decentralized Inventory Control Systemscited by 10

50 most cited of 281 All 281 (JSON)