mlsanityData to 14 Oct 2026

GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS

arXiv 2408.01584 ↗First posted 2 Aug 2024cs.AI cs.AR cs.GRcited by 45

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Multi-agent learning algorithms have been successful at generating superhuman planning in various games but have had limited impact on the design of deployed multi-agent planners. A key bottleneck in applying these techniques to multi-agent planning is that they require billions of steps of experience. To enable the study of multi-agent planning at scale, we present GPUDrive. GPUDrive is a GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine capable of generating over a million simulation steps per second. Observation, reward, and dynamics functions are written directly in C++, allowing users to define complex, heterogeneous agent behaviors that are lowered to high-performance CUDA. Despite these low-level optimizations, GPUDrive is fully accessible through Python, offering a seamless and efficient workflow for multi-agent, closed-loop simulation. Using GPUDrive, we train reinforcement learning agents on the Waymo Open Motion Dataset, achieving efficient goal-reaching in minutes and scaling to thousands of scenarios in hours. We open-source the code and pre-trained agents at https://github.com/Emerge-Lab/gpudrive.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 25 of its 46 references that are in the map. First posted on arXiv 2 Aug 2024.

Builds on

25 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research12 Oct
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec
  3. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving18 Oct

Other references · 22

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  2. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  3. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  4. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  5. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  6. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  7. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  8. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  9. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 68
  10. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  11. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  12. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
10 more
  1. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  2. 2022Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planningcited by 48
  3. 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 40
  4. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32
  5. 2017Sim4CV: A Photo-Realistic Simulator for Computer Vision Applicationscited by 31
  6. 2022InterSim: Interactive Traffic Simulation via Explicit Relation Modelingcited by 27
  7. 2017Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Carscited by 26
  8. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 24
  9. 2021DriverGym: Democratising Reinforcement Learning for Autonomous Drivingcited by 23
  10. 2020MADRaS : Multi Agent Driving Simulatorcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 23.

45 papers.

Sort:

By area:

  1. 2025Pseudo-Simulation for Autonomous Drivingcited by 150
  2. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  3. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  4. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 28
  5. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 23
  6. 2025Building reliable sim driving agents by scaling self-playcited by 21
  7. 2025Towards Autonomous Micromobility through Scalable Urban Simulationcited by 16
  8. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  9. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  10. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  11. 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Drivingcited by 5
  12. 2024Crafting Desirable Climate Trajectories with RL Explored Socio-Environmental Simulationscited by 4
33 more
  1. 2025Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Drivingcited by 4
  2. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  3. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  4. 2025V-Max: A Reinforcement Learning Framework for Autonomous Drivingcited by 3
  5. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  6. 2024Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulationcited by 2
  7. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  8. 2025Estimating cognitive biases with attention-aware inverse planningcited by 2
  9. 2025Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscalecited by 2
  10. 2026Heterogeneous Self-Play for Realistic Highway Traffic Simulationcited by 2
  11. 2026Benchmarking Open-Ended Multi-Agent Coordination in Language Agentscited by 2
  12. 2026IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarkingcited by 2
  13. 2026Pictura: Perspective-View Self-Play at Scale for Drivingcited by 2
  14. 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oraclescited by 1
  15. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
  16. 2026Constrained Meta Reinforcement Learning with Provable Test-Time Safetycited by 1
  17. 2026HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Trafficcited by 1
  18. 2026TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scalecited by 1
  19. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  20. 2026OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teachercited by 1
  21. 2026CounterPlay: Counterfactual Post-Training for Self-Play Driving Policiescited by 1
  22. 2025Ludax: A GPU-Accelerated Domain Specific Language for Board Gamescited by 0
  23. 2025Assistax: A Multi-Agent Hardware-Accelerated Reinforcement Learning Benchmark for Assistive Roboticscited by 0
  24. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  25. 2026ScenarioControl: Vision-Language Controllable Vectorized Latent Scenario Generationcited by 0
  26. 2026Dynamics Distillation for Efficient and Transferable Control Learningcited by 0
  27. 2026FAST: A Framework for Aligned Sampling and Training in Parallel Reinforcement Learning for Autonomous Drivingcited by 0
  28. 2026What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planningcited by 0
  29. 2026Threat-guided Policy-aware Scene Perturbation for Safe Autonomous Driving with Online Reinforcement Learningcited by 0
  30. 2026rl-triton: High-Performance Triton GPU Kernels for Reinforcement Learning Credit Assignmentcited by 0
  31. 2026Comfort by Construction: Adaptive, Comfort-Bounded Action Spaces for Learned Driving Policiescited by 0
  32. 2026DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Drivingcited by 0
  33. 2026TerrainForge: Physics-Grounded Road Geometry Editing for Counterfactual Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026G-MAPP: GPU-accelerated Multi-Agent Planning and Perception for Reactive Motion Generation ↗
  2. 2026Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning
  3. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
  4. 2024PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators
  5. 2026Mahjax: A GPU-Accelerated Mahjong Simulator for Reinforcement Learning in JAX
  6. 2026StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
  7. 2026TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
  8. 2025Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning