# GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS

arXiv: https://arxiv.org/abs/2408.01584 · HTML: https://mlsanity.com/paper/2408.01584/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 25 of 25)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research](https://mlsanity.com/paper/2310.08710/) (2023)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios](https://mlsanity.com/paper/2212.11419/) (2022)
- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [BITS: Bi-level Imitation for Traffic Simulation](https://mlsanity.com/paper/2208.12403/) (2022)
- [VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning](https://mlsanity.com/paper/2207.03530/) (2022)
- [JaxMARL: Multi-Agent RL Environments and Algorithms in JAX](https://mlsanity.com/paper/2311.10090/) (2023)
- [Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2306.09884/) (2023)
- [Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world](https://mlsanity.com/paper/2206.09889/) (2022)
- [Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1911.04175/) (2019)
- [Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning](https://mlsanity.com/paper/2210.05492/) (2022)
- [Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving](https://mlsanity.com/paper/2210.09539/) (2022)
- [SUMMIT: A Simulator for Urban Driving in Massive Mixed Traffic](https://mlsanity.com/paper/1911.04074/) (2019)
- [Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation](https://mlsanity.com/paper/2104.11212/) (2021)
- [Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications](https://mlsanity.com/paper/1708.05869/) (2017)
- [InterSim: Interactive Traffic Simulation via Explicit Relation Modeling](https://mlsanity.com/paper/2210.14413/) (2022)
- [Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars](https://mlsanity.com/paper/1712.01397/) (2017)
- [PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice](https://mlsanity.com/paper/2406.12905/) (2024)
- [DriverGym: Democratising Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2111.06889/) (2021)
- [MADRaS : Multi Agent Driving Simulator](https://mlsanity.com/paper/2010.00993/) (2020)

## Built on by (most cited first, 30 of 45)

- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks](https://mlsanity.com/paper/2410.23208/) (2024)
- [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025)
- [Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments](https://mlsanity.com/paper/2503.22496/) (2025)
- [LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving](https://mlsanity.com/paper/2512.20563/) (2025)
- [Building reliable sim driving agents by scaling self-play](https://mlsanity.com/paper/2502.14706/) (2025)
- [Towards Autonomous Micromobility through Scalable Urban Simulation](https://mlsanity.com/paper/2505.00690/) (2025)
- [SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer](https://mlsanity.com/paper/2509.18648/) (2025)
- [SPACeR: Self-Play Anchoring with Centralized Reference Models](https://mlsanity.com/paper/2510.18060/) (2025)
- [LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Grounding](https://mlsanity.com/paper/2508.19204/) (2025)
- [Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving](https://mlsanity.com/paper/2605.10034/) (2026)
- [Crafting Desirable Climate Trajectories with RL Explored Socio-Environmental Simulations](https://mlsanity.com/paper/2410.07287/) (2024)
- [Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving](https://mlsanity.com/paper/2510.15505/) (2025)
- [Human-like autonomy emerges from self-play and a pinch of human data](https://mlsanity.com/paper/2606.19370/) (2026)
- [Scaling Self-Play for End-to-End Driving](https://mlsanity.com/paper/2606.19641/) (2026)
- [V-Max: A Reinforcement Learning Framework for Autonomous Driving](https://mlsanity.com/paper/2503.08388/) (2025)
- [Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges](https://mlsanity.com/paper/2512.00021/) (2025)
- [Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulation](https://mlsanity.com/paper/2412.16750/) (2024)
- [SceneStreamer: Continuous Scenario Generation as Next Token Group Prediction](https://mlsanity.com/paper/2506.23316/) (2025)
- [Estimating cognitive biases with attention-aware inverse planning](https://mlsanity.com/paper/2510.25951/) (2025)
- [Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale](https://mlsanity.com/paper/2511.04904/) (2025)
- [Heterogeneous Self-Play for Realistic Highway Traffic Simulation](https://mlsanity.com/paper/2604.16406/) (2026)
- [Benchmarking Open-Ended Multi-Agent Coordination in Language Agents](https://mlsanity.com/paper/2606.08340/) (2026)
- [IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarking](https://mlsanity.com/paper/2606.08729/) (2026)
- [Pictura: Perspective-View Self-Play at Scale for Driving](https://mlsanity.com/paper/2607.26005/) (2026)
- [TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles](https://mlsanity.com/paper/2408.15538/) (2024)
- [Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation](https://mlsanity.com/paper/2512.05812/) (2025)
- [Constrained Meta Reinforcement Learning with Provable Test-Time Safety](https://mlsanity.com/paper/2601.21845/) (2026)
- [HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic](https://mlsanity.com/paper/2602.03447/) (2026)
- [TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale](https://mlsanity.com/paper/2607.13028/) (2026)

## Abstract (arXiv)

Multi-agent learning algorithms have been successful at generating superhuman planning in various games but have had limited impact on the design of deployed multi-agent planners. A key bottleneck in applying these techniques to multi-agent planning is that they require billions of steps of experience. To enable the study of multi-agent planning at scale, we present GPUDrive. GPUDrive is a GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine capable of generating over a million simulation steps per second. Observation, reward, and dynamics functions are written directly in C++, allowing users to define complex, heterogeneous agent behaviors that are lowered to high-performance CUDA. Despite these low-level optimizations, GPUDrive is fully accessible through Python, offering a seamless and efficient workflow for multi-agent, closed-loop simulation. Using GPUDrive, we train reinforcement learning agents on the Waymo Open Motion Dataset, achieving efficient goal-reaching in minutes and scaling to thousands of scenarios in hours. We open-source the code and pre-trained agents at https://github.com/Emerge-Lab/gpudrive.