mlsanityData to 7 Oct 2026

Robust Autonomy Emerges from Self-Play

arXiv 2502.03349 ↗First posted 5 Feb 2025cs.LG cs.AI cs.ROcited by 64

Not on the map. Why?

Cited by 33 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 45 of its 73 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

45 of 73 references.

References · 45

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2017Deep Setscited by 1,394
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  6. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  7. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  10. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  11. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  12. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
33 more
  1. 2017Population Based Training of Neural Networkscited by 351
  2. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  3. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  4. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  5. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  6. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  7. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  8. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  9. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  10. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  11. 2022Learning from All Vehiclescited by 135
  12. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  13. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  14. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  15. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  16. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  17. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  18. 2023The Waymo Open Sim Agents Challengecited by 86
  19. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  20. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  21. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  22. 2023From Prediction to Planning With Goal Conditioned Lane Graph Traversalscited by 45
  23. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  24. 2016Learning to Act by Predicting the Futurecited by 36
  25. 2021Autonomy 2.0: Why is self-driving always 5 years away?cited by 30
  26. 2023Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023cited by 25
  27. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 24
  28. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  29. 2023SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Modelscited by 24
  30. 2021Megaverse: Simulating Embodied Agents at One Million Experiences per Secondcited by 19
  31. 2021Large Batch Simulation for Deep Reinforcement Learningcited by 18
  32. 2020REPAINT: Knowledge Transfer in Deep Reinforcement Learningcited by 14
  33. 2021Reimagining an autonomous vehiclecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 31; 2026: 33.

64 papers.

Sort:

By area:

  1. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learningcited by 84
  2. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  3. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  4. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  5. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  6. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Drivingcited by 24
  7. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  8. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 22
  9. 2025Building reliable sim driving agents by scaling self-playcited by 20
  10. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 19
  11. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  12. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
38 more
  1. 2026Fail2Drive: Benchmarking Closed-Loop Driving Generalizationcited by 15
  2. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  3. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  4. 2025Scalable Decision Making for Games of Imperfect Informationcited by 7
  5. 2026Digital Red Queen: Adversarial Program Evolution in Core War with LLMscited by 7
  6. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 6
  7. 2025Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Modelscited by 5
  8. 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Drivingcited by 5
  9. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  10. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
  11. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  12. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  13. 2025V-Max: A Reinforcement Learning Framework for Autonomous Drivingcited by 3
  14. 2025Good Actions Succeed, Bad Actions Generalize: A Case Study on Why RL Generalizes Bettercited by 3
  15. 2025Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedbackcited by 3
  16. 2026TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrationscited by 3
  17. 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
  18. 2025Agile Flight Emerges from Multi-Agent Competitive Racingcited by 2
  19. 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Drivingcited by 2
  20. 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
  21. 2026Heterogeneous Self-Play for Realistic Highway Traffic Simulationcited by 2
  22. 2026123D: Unifying Multi-Modal Autonomous Driving Data at Scalecited by 2
  23. 2026Pictura: Perspective-View Self-Play at Scale for Drivingcited by 2
  24. 2025Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Surveycited by 1
  25. 2025Multiple-Frequencies Population-Based Trainingcited by 1
  26. 2025Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learningcited by 1
  27. 2025RDAR: Reward-Driven Agent Relevance Estimation for Autonomous Drivingcited by 1
  28. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
  29. 2026Reward-Conditioned Reinforcement Learningcited by 1
  30. 2026PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learningcited by 1
  31. 2026Efficient Equivariant Transformer for Self-Driving Agent Modelingcited by 1
  32. 2026Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulationcited by 1
  33. 2026MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Drivingcited by 1
  34. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  35. 2026What Emerges and What Breaks in Self-Play Drivingcited by 1
  36. 2026OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teachercited by 1
  37. 2025TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learningcited by 0
  38. 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environmentscited by 0

50 most cited of 64 All 64 (JSON)