# Mava: a research library for distributed multi-agent reinforcement learning in JAX

arXiv: https://arxiv.org/abs/2107.01460 · HTML: https://mlsanity.com/paper/2107.01460/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 3 Jul 2021.

## Builds on (most cited first, 10 of 10)

- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [FACMAC: Factored Multi-Agent Centralised Policy Gradients](https://mlsanity.com/paper/2003.06709/) (2020)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.03400/) (2021)
- [A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systems](https://mlsanity.com/paper/1506.01170/) (2015)
- [Podracer architectures for scalable Reinforcement Learning](https://mlsanity.com/paper/2104.06272/) (2021)

## Built on by (most cited first, 16 of 16)

- [BenchMARL: Benchmarking Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2312.01472/) (2023)
- [WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU](https://mlsanity.com/paper/2108.13976/) (2021)
- [PantheonRL: A MARL Library for Dynamic Training Interactions](https://mlsanity.com/paper/2112.07013/) (2021)
- [Sable: a Performant, Efficient and Scalable Sequence Model for MARL](https://mlsanity.com/paper/2410.01706/) (2024)
- [Pearl: A Production-ready Reinforcement Learning Agent](https://mlsanity.com/paper/2312.03814/) (2023)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Multi-Agent Guided Policy Optimization](https://mlsanity.com/paper/2507.18059/) (2025)
- [A Reference Architecture of Reinforcement Learning Frameworks](https://mlsanity.com/paper/2603.06413/) (2026)
- [Multi-Agent Reinforcement Learning with Selective State-Space Models](https://mlsanity.com/paper/2410.19382/) (2024)
- [Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies](https://mlsanity.com/paper/2505.21236/) (2025)
- [marl-jax: Multi-Agent Reinforcement Leaning Framework](https://mlsanity.com/paper/2303.13808/) (2023)
- [EconoJax: A Fast & Scalable Economic Simulation in Jax](https://mlsanity.com/paper/2410.22165/) (2024)
- [EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning](https://mlsanity.com/paper/2501.15129/) (2025)
- [Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.09212/) (2026)
- [Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation](https://mlsanity.com/paper/2605.13554/) (2026)
- [MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL](https://mlsanity.com/paper/2610.14883/) (2026)

## Abstract (arXiv)

Multi-agent reinforcement learning (MARL) research is inherently computationally expensive and it is often difficult to obtain a sufficient number of experiment samples to test hypotheses and make robust statistical claims. Furthermore, MARL algorithms are typically complex in their design and can be tricky to implement correctly. These aspects of MARL present a difficult challenge when it comes to creating useful software for advanced research. Our criteria for such software is that it should be simple enough to use to implement new ideas quickly, while at the same time be scalable and fast enough to test those ideas in a reasonable amount of time. In this preliminary technical report, we introduce Mava, a research library for MARL written purely in JAX, that aims to fulfill these criteria. We discuss the design and core features of Mava, and demonstrate its use and performance across a variety of environments. In particular, we show Mava's substantial speed advantage, with improvements of 10-100x compared to other popular MARL frameworks, while maintaining strong performance. This allows for researchers to test ideas in a few minutes instead of several hours. Finally, Mava forms part of an ecosystem of libraries that seamlessly integrate with each other to help facilitate advanced research in MARL. We hope Mava will benefit the community and help drive scientifically sound and statistically robust research in the field. The open-source repository for Mava is available at https://github.com/instadeepai/Mava.