mlsanityData to 14 Oct 2026

MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

arXiv 2506.14990 ↗First posted 17 Jun 2025cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Benchmarks play a central role in reinforcement learning (RL) research, yet their computational constraints often shape what is studied. Despite the motivation of lifelong learning, most continual RL papers consider only 3-10 sequential tasks, as CPU-bound environments make longer sequences impractical. Meanwhile, continual learning in cooperative multi-agent settings remains largely unexplored. To address these gaps, we introduce MEAL (Multi-agent Environments for Adaptive Learning), the first benchmark for continual multi-agent RL. By leveraging JAX and GPU acceleration, MEAL enables training on sequences of 100 tasks in a few hours on a single GPU. We find that long task sequences reveal failure modes that do not appear at smaller scales.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 54 of its 75 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

54 of 75 references.

References · 54

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  4. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  5. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  9. 2019The StarCraft Multi-Agent Challengecited by 597
  10. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  11. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  12. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
38 more
  1. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  2. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  3. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  4. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  5. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  6. 2017Learning with Opponent-Learning Awarenesscited by 222
  7. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 219
  8. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  9. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  10. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  11. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  12. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  13. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  14. 2021Collaborating with Humans without Human Datacited by 142
  15. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  16. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  17. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  18. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  19. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  20. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  21. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  22. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 69
  23. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  24. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
  25. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  26. 2022Melting Pot 2.0cited by 40
  27. 2023A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environmentcited by 39
  28. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
  29. 2020Flatland-RL : Multi-Agent Reinforcement Learning on Trainscited by 32
  30. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 32
  31. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 31
  32. 2018Pommerman: A Multi-Agent Playgroundcited by 30
  33. 2021Evaluating the Robustness of Collaborative Agentscited by 25
  34. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  35. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  36. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 11
  37. 2025SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 8
  38. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2025A Survey of Continual Reinforcement Learningcited by 18
  2. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  3. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  4. 2025Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Reviewcited by 1
  5. 2026Task diversity produces systematic transfer but inhibits continual reinforcement learningcited by 0
  6. 2026Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methodscited by 0
  7. 2026A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturingcited by 0
  8. 2026Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVRcited by 0
  9. 2026Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learningcited by 0
  10. 2026BluffJAX: Adversarial Imperfect Information Games in JAXcited by 0
  11. 2026Continual Graph Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning
  2. 2025A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks
  3. 2024Continual Task Learning through Adaptive Policy Self-Composition
  4. 2025An Empirical Study of Deep Reinforcement Learning in Continuing Tasks
  5. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  6. 2026Position: Deployed Reinforcement Learning should be Continual
  7. 2026Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning
  8. 2025Continual Learning, Not Training: Online Adaptation For Agents