mlsanityData to 7 Oct 2026

PettingZoo: Gym for Multi-Agent Reinforcement Learning

arXiv 2009.14471 ↗First posted 30 Sep 2020cs.LG cs.MA stat.MLcited by 209

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 20; 2023: 35; 2024: 38; 2025: 49; 2026: 48.

209 papers.

Sort:

By area:

  1. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  2. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  3. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  4. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  5. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  6. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  7. 2023Principles and Guidelines for Evaluating Social Robot Navigation Algorithmscited by 52
  8. 2023Lyfe Agents: Generative agents for low-cost real-time social interactionscited by 49
  9. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 45
  10. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  11. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  12. 2024LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environmentscited by 35
38 more
  1. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 35
  2. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  3. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  4. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  5. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  6. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 22
  7. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  8. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  9. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  10. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggerscited by 19
  11. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  12. 2024Zero-Shot Reinforcement Learning via Function Encoderscited by 17
  13. 2022LibSignal: An Open Library for Traffic Signal Controlcited by 16
  14. 2021PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systemscited by 15
  15. 2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
  16. 2023Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilitiescited by 15
  17. 2022Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teamingcited by 14
  18. 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attackscited by 14
  19. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  20. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learningcited by 13
  21. 2021GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Managementcited by 12
  22. 2022A Survey on Reinforcement Learning Methods in Character Animationcited by 12
  23. 2021Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learningcited by 11
  24. 2021On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learningcited by 10
  25. 2021PantheonRL: A MARL Library for Dynamic Training Interactionscited by 10
  26. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  27. 2024Verifiably Robust Conformal Predictioncited by 10
  28. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learningcited by 10
  29. 2023TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problemscited by 9
  30. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  31. 2023Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learningcited by 9
  32. 2024A Review of Nine Physics Engines for Reinforcement Learning Researchcited by 9
  33. 2021Learning Altruistic Behaviours in Reinforcement Learning without External Rewardscited by 8
  34. 2024Language Models are Spacecraft Operatorscited by 8
  35. 2022Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systemscited by 7
  36. 2023Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learningcited by 7
  37. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  38. 2025WFCRL: A Multi-Agent Reinforcement Learning Benchmark for Wind Farm Controlcited by 7

50 most cited of 209 All 209 (JSON)