mlsanityData to 7 Oct 2026

BenchMARL: Benchmarking Multi-Agent Reinforcement Learning

arXiv 2312.01472 ↗First posted 3 Dec 2023cs.LG cs.AI cs.MAcited by 31

Not on the map. Why?

Abstract (arXiv)

The field of Multi-Agent Reinforcement Learning (MARL) is currently facing a reproducibility crisis. While solutions for standardized reporting have been proposed to address the issue, we still lack a benchmarking tool that enables standardization and reproducibility, while leveraging cutting-edge Reinforcement Learning (RL) implementations. In this paper, we introduce BenchMARL, the first MARL training library created to enable standardized benchmarking across different algorithms, models, and environments. BenchMARL uses TorchRL as its backend, granting it high performance and maintained state-of-the-art implementations while addressing the broad community of MARL PyTorch users. Its design enables systematic configuration and reporting, thus allowing users to create and run complex benchmarks from simple one-line inputs. BenchMARL is open-sourced on GitHub: https://github.com/facebookresearch/BenchMARL

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

22 of 31 references.

References · 22

  1. 2017Graph Attention Networkscited by 8,854
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2017Deep Setscited by 1,394
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2019The StarCraft Multi-Agent Challengecited by 589
  9. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  10. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  11. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  12. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
10 more
  1. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  2. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  3. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  4. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  5. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  6. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  7. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  8. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  9. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  10. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 18; 2026: 7.

31 papers.

Sort:

By area:

  1. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  2. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  3. 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 17
  4. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  5. 2025Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Controlcited by 7
  6. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domainscited by 6
  7. 2024Representation Learning For Efficient Deep Multi-Agent Reinforcement Learningcited by 3
  8. 2024InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemmacited by 3
  9. 2024The impact of behavioral diversity in multi-agent reinforcement learningcited by 3
  10. 2025URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehiclescited by 3
  11. 2026A Reference Architecture of Reinforcement Learning Frameworkscited by 3
  12. 2025TAG: A Decentralized Framework for Multi-Agent Hierarchical Reinforcement Learningcited by 2
19 more
  1. 2025LLM-Enhanced Multi-Agent Reinforcement Learning with Expert Workflow for Real-Time P2P Energy Tradingcited by 2
  2. 2025CAMAR: Continuous Actions Multi-Agent Routingcited by 2
  3. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  4. 2026Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineeringcited by 2
  5. 2025A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inferencecited by 1
  6. 2025JaxRobotarium: Training and Deploying Multi-Robot Policies in 10 Minutescited by 1
  7. 2025Curriculum Imitation Learning of Distributed Multi-Robot Policiescited by 1
  8. 2026Randomness is sometimes necessary for coordinationcited by 1
  9. 2024FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RLcited by 0
  10. 2024Multi-Agent Environments for Vehicle Routing Problemscited by 0
  11. 2025Moving Out: Physically-grounded Human-AI Collaborationcited by 0
  12. 2025ReCoDe: Reinforcement Learning-based Dynamic Constraint Design for Multi-Agent Coordinationcited by 0
  13. 2025The Heterogeneous Multi-Agent Challengecited by 0
  14. 2025Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning taskscited by 0
  15. 2025Semi-overlapping Multi-bandit Best Arm Identification for Sequential Support Network Learningcited by 0
  16. 2026MOSAIC: A Universal Agent-Level Interface for Cross-Paradigm Agent Mixing and Human-AI Collaborationcited by 0
  17. 2026Game-Theory-Assisted Reinforcement Learning for Border Defense: Early Termination based on Analytical Solutionscited by 0
  18. 2026PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillationcited by 0
  19. 2026Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learningcited by 0