mlsanityData to 7 Oct 2026

Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation

arXiv 2106.13281 ↗First posted 24 Jun 2021cs.RO cs.AIcited by 375

Not on the map. Why?

Abstract (arXiv)

We present Brax, an open source library for rigid body simulation with a focus on performance and parallelism on accelerators, written in JAX. We present results on a suite of tasks inspired by the existing reinforcement learning literature, but remade in our engine. Additionally, we provide reimplementations of PPO, SAC, ES, and direct policy optimization in JAX that compile alongside our environments, allowing the learning algorithm and the environment processing to occur on the same device, and to scale seamlessly on accelerators. Finally, we include notebooks that facilitate training of performant policies on common OpenAI Gym MuJoCo-like tasks in minutes.

Key facts. Cited by 375 papers in the map (data through 7 Oct 2026). Builds on 23 of its 57 references that are in the map. First posted on arXiv 24 Jun 2021.

Builds on

23 of 57 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2018DeepMind Control Suitecited by 935
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  10. 2019Solving Rubik's Cube with a Robot Handcited by 699
  11. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  12. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
11 more
  1. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 248
  2. 2018Unity: A General Platform for Intelligent Agentscited by 232
  3. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  4. 2016A Differentiable Physics Engine for Deep Learning in Roboticscited by 107
  5. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  6. 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 77
  7. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  8. 2021Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contactcited by 50
  9. 2017Unbiasing Truncated Backpropagation Through Timecited by 37
  10. 2021Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarkingcited by 10
  11. 2020Lyceum: An efficient and scalable ecosystem for robot learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 52; 2023: 66; 2024: 70; 2025: 83; 2026: 88.

375 papers.

Sort:

By area:

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  2. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  3. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  4. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  5. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  6. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  7. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  8. 2022Discovered Policy Optimisationcited by 103
  9. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 92
  10. 2025Flow Matching Policy Gradientscited by 86
  11. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 83
  12. 2021Gradients are Not All You Needcited by 82
38 more
  1. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  2. 2022AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutionscited by 77
  3. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  4. 2025Robust Autonomy Emerges from Self-Playcited by 64
  5. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  6. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  7. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  8. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 58
  9. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  10. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  11. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  12. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 52
  13. 2022NeuPhysics: Editable Neural Geometry and Physics from Monocular Videoscited by 50
  14. 2022Differentiable Physics Simulation of Dynamics-Augmented Neural Objectscited by 45
  15. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 45
  16. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  17. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  18. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  19. 2021Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modelingcited by 40
  20. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  21. 2022evosax: JAX-based Evolution Strategiescited by 39
  22. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilitiescited by 39
  23. 2022Differentiable Dynamics for Articulated 3d Human Motion Reconstructioncited by 38
  24. 2022Physical Design using Differentiable Learned Simulatorscited by 37
  25. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 35
  26. 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 34
  27. 2023Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimizationcited by 33
  28. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 33
  29. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  30. 2024Model-Based Diffusion for Trajectory Optimizationcited by 30
  31. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  32. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  33. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  34. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 24
  35. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  36. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
  37. 2023Benchmarking Offline Reinforcement Learning on Real-Robot Hardwarecited by 23
  38. 2023InsActor: Instruction-driven Physics-based Characterscited by 23

50 most cited of 375 All 375 (JSON)