# TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale arXiv: https://arxiv.org/abs/2607.13028 ยท HTML: https://mlsanity.com/paper/2607.13028/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. - Builds on 38 of its 54 references that are in the map. - First posted on arXiv 14 Jul 2026. ## Placed in - End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/) - Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/) ## Builds on (most cited first, 30 of 38) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015) - [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017) - [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017) - [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019) - [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015) - [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018) - [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021) - [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021) - [NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking](https://mlsanity.com/paper/2406.15349/) (2024) - [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018) - [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023) - [GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving](https://mlsanity.com/paper/2303.05760/) (2023) - [Diffusion-Based Planning for Autonomous Driving with Flexible Guidance](https://mlsanity.com/paper/2501.15564/) (2025) - [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021) - [Emergent Complexity via Multi-Agent Competition](https://mlsanity.com/paper/1710.03748/) (2017) - [The Waymo Open Sim Agents Challenge](https://mlsanity.com/paper/2305.12032/) (2023) - [Learning values across many orders of magnitude](https://mlsanity.com/paper/1602.07714/) (2016) - [PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving](https://mlsanity.com/paper/2404.14327/) (2024) - [Time Limits in Reinforcement Learning](https://mlsanity.com/paper/1712.00378/) (2017) - [SimNet: Learning Reactive Self-driving Simulations from Real-world Observations](https://mlsanity.com/paper/2105.12332/) (2021) - [Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world](https://mlsanity.com/paper/2206.09889/) (2022) - [TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction](https://mlsanity.com/paper/2303.04116/) (2023) - [DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving](https://mlsanity.com/paper/2310.05885/) (2023) - [From Prediction to Planning With Goal Conditioned Lane Graph Traversals](https://mlsanity.com/paper/2302.07753/) (2023) - [GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS](https://mlsanity.com/paper/2408.01584/) (2024) - [Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?](https://mlsanity.com/paper/2404.07569/) (2024) - [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025) - [Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation](https://mlsanity.com/paper/2104.11212/) (2021) ## Built on by (most cited first, 1 of 1) - [INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement](https://mlsanity.com/paper/2609.31137/) (2026) ## Abstract (arXiv) Training robust autonomous driving agents requires a simulator fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains. We present TerraZero, a procedural driving simulator and self-play training stack that meets these goals. A configurable C engine runs simulation on the CPU and policy inference on the GPU over a zero-copy path, sustaining 1.3M agent-steps per second on a single server-grade GPU, far faster than existing object-level simulators, while keeping fidelity lighter single-agent systems omit: heterogeneous agents, multiple dynamics models, and full traffic-rule enforcement. TerraZero uses logged data only as a source of real-world map geometry, populating each map with randomized rule-based road users and signal controllers and randomizing agent dynamics, rewards, and sizes per episode, so one map yields an effectively unbounded set of scenarios. Every reported policy trains from scratch by reinforcement learning alone, with zero human demonstrations, no imitation, no logged trajectories, and no fallback planner at inference, on a compute-efficient self-play recipe scaled across GPUs. The policies generalize zero-shot across cities and datasets, including emergent left-hand-traffic driving without explicit supervision. As an ego policy, a single checkpoint is, to our knowledge, the first fully learned policy to top both val14 and the interactive long-tail InterPlan suite. On Waymo Open Sim Agents realism the same recipe outperforms other demonstration-free methods and is competitive with the strongest reference-anchored self-play method. One stack serves both roles: state-of-the-art demonstration-free driving policies across dynamics for cars and trucks, and sim agents that jointly control vehicles, pedestrians, and cyclists.