mlsanityData to 14 Oct 2026

Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale

arXiv 2511.04904 ↗First posted 7 Nov 2025cs.LG cs.MAcited by 2

Not on the map. Why?

Abstract (arXiv)

Progress in multi-agent reinforcement learning (MARL) requires challenging benchmarks that assess the limits of current methods. However, existing benchmarks often target narrow short-horizon challenges that do not adequately stress the long-term dependencies and generalization capabilities inherent in many multi-agent systems. To address this, we first present textitCraftax-MA: an extension of the popular open-ended RL environment, Craftax, that supports multiple agents and evaluates a wide range of general abilities within a single environment. Written in JAX, textitCraftax-MA is exceptionally fast with a training run using 250 million environment interactions completing in under an hour. To provide a more compelling challenge for MARL, we also present textitCraftax-Coop, an extension introducing heterogeneous agents, trading and more mechanics that require complex cooperation among agents for success. We provide analysis demonstrating that existing algorithms struggle with key challenges in this benchmark, including long-horizon credit assignment, exploration and cooperation, and argue for its potential to drive long-term research in MARL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 7 Nov 2025.

Builds on

29 of 47 references.

References · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  3. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  4. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  5. 2019The StarCraft Multi-Agent Challengecited by 597
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  8. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  9. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  10. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  11. 2020The NetHack Learning Environmentcited by 198
  12. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
17 more
  1. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  2. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  3. 2022Discovered Policy Optimisationcited by 104
  4. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  5. 2024Simplifying Deep Temporal Difference Learningcited by 78
  6. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 68
  7. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  8. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  9. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  10. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 51
  11. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 45
  12. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  13. 2019Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agentscited by 44
  14. 2021Podracer architectures for scalable Reinforcement Learningcited by 38
  15. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 24
  16. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 18
  17. 2024XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A Recipe for Stable Offline Multi-agent Reinforcement Learningcited by 2
  2. 2026Benchmarking Open-Ended Multi-Agent Coordination in Language Agentscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025An Extended Benchmarking of Multi-Agent Reinforcement Learning Algorithms in Complex Fully Cooperative Tasks
  2. 2026MARS2: Scaling Multi-Agent Tree Search via Reinforcement Learning for Code Generation
  3. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
  4. 2026Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning
  5. 2024MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search
  6. 2025Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks
  7. 2026Claw-R1: A Step-Level Data Middleware System for Agentic Reinforcement Learning
  8. 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation