mlsanityData to 14 Oct 2026

minimax: Efficient Baselines for Autocurricula in JAX

arXiv 2311.12716 ↗First posted 21 Nov 2023cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Unsupervised environment design (UED) is a form of automatic curriculum learning for training robust decision-making agents to zero-shot transfer into unseen environments. Such autocurricula have received much interest from the RL community. However, UED experiments, based on CPU rollouts and GPU model updates, have often required several weeks of training. This compute requirement is a major obstacle to rapid innovation for the field. This work introduces the minimax library for UED training on accelerated hardware. Using JAX to implement fully-tensorized environments and autocurriculum algorithms, minimax allows the entire training loop to be compiled for hardware acceleration. To provide a petri dish for rapid experimentation, minimax includes a tensorized grid-world based on MiniGrid, in addition to reusable abstractions for conducting autocurricula in procedurally-generated environments. With these components, minimax provides strong UED baselines, including new parallelized variants, which achieve over 120× speedups in wall time compared to previous implementations when training with equal batch sizes. The minimax library is available under the Apache 2.0 license at https://github.com/facebookresearch/minimax.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 21 Nov 2023.

Builds on

49 of 64 references.

References · 49

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Layer Normalizationcited by 5,669
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  9. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  11. 2017Population Based Training of Neural Networkscited by 367
  12. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
37 more
  1. 2018An Empirical Model of Large-Batch Trainingcited by 281
  2. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  4. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  5. 2017Teacher-Student Curriculum Learningcited by 210
  6. 2020Prioritized Level Replaycited by 198
  7. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  8. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  9. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  10. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  11. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  12. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  13. 2019Behaviour Suite for Reinforcement Learningcited by 132
  14. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  15. 2021Replay-Guided Adversarial Environment Designcited by 111
  16. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  17. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  18. 2022Discovered Policy Optimisationcited by 104
  19. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  20. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  21. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  22. 2021Off-Belief Learningcited by 75
  23. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  24. 2019Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Researchcited by 61
  25. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  26. 2019TorchBeast: A PyTorch Platform for Distributed RLcited by 41
  27. 2022evosax: JAX-based Evolution Strategiescited by 41
  28. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  29. 2021Podracer architectures for scalable Reinforcement Learningcited by 38
  30. 2020BeBold: Exploration Beyond the Boundary of Explored Regionscited by 33
  31. 2020Griddly: A platform for AI research in gamescited by 31
  32. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 31
  33. 2022EvoJAX: Hardware-Accelerated Neuroevolutioncited by 29
  34. 2021Megaverse: Simulating Embodied Agents at One Million Experiences per Secondcited by 20
  35. 2023Stabilizing Unsupervised Environment Design with a Learned Adversarycited by 17
  36. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  37. 2023JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for tradingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 2; 2026: 1.

10 papers.

  1. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  2. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  3. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 31
  4. 2024Refining Minimax Regret for Unsupervised Environment Designcited by 18
  5. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 18
  6. 2024JaxUED: A simple and useable UED library in Jaxcited by 11
  7. 2024The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Designcited by 11
  8. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environmentscited by 4
  9. 2025EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learningcited by 1
  10. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1