mlsanityData to 14 Oct 2026

Building reliable sim driving agents by scaling self-play

arXiv 2502.14706 ↗First posted 20 Feb 2025cs.AI cs.ROcited by 21

Not on the map. Why?

Abstract (arXiv)

Simulation agents are essential for designing and testing systems that interact with humans, such as autonomous vehicles (AVs). These agents serve various purposes, from benchmarking AV performance to stress-testing system limits, but all applications share one key requirement: reliability. To enable sound experimentation, a simulation agent must behave as intended. It should minimize actions that may lead to undesired outcomes, such as collisions, which can distort the signal-to-noise ratio in analyses. As a foundation for reliable sim agents, we propose scaling self-play to thousands of scenarios on the Waymo Open Motion Dataset under semi-realistic limits on human perception and control. Training from scratch on a single GPU, our agents solve almost the full training set within a day. They generalize to unseen test scenes, achieving a 99.8% goal completion rate with less than 0.8% combined collision and off-road incidents across 10,000 held-out scenarios. Beyond in-distribution generalization, our agents show partial robustness to out-of-distribution scenes and can be fine-tuned in minutes to reach near-perfect performance in such cases. We open-source the pre-trained agents and integrate them with a batched multi-agent simulator. Demonstrations of agent behaviors can be viewed at https://sites.google.com/view/reliable-sim-agents, and we open-source our agents at https://github.com/Emerge-Lab/gpudrive.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

26 of 42 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  6. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  7. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  8. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  9. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  10. 2023The Waymo Open Sim Agents Challengecited by 86
  11. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  12. 2020A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systemscited by 71
14 more
  1. 2025Robust Autonomy Emerges from Self-Playcited by 67
  2. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 60
  3. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  4. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 50
  5. 2022Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planningcited by 48
  6. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 45
  7. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  8. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  9. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  10. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  11. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  12. 2024UniGen: Unified Modeling of Initial Agent States and Trajectories for Generating Autonomous Driving Scenarioscited by 10
  13. 2024Data Scaling Laws for Imitation Learning-Based End-to-End Autonomous Drivingcited by 10
  14. 2022Modeling road user response timing in naturalistic settings: a surprise-based frameworkcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 14.

21 papers.

Sort:

By area:

  1. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
  2. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  3. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  4. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 7
  5. 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Drivingcited by 5
  6. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  7. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  8. 2026DriveZero: End-to-End Driving Beyond Human Demonstrationscited by 3
  9. 2025Estimating cognitive biases with attention-aware inverse planningcited by 2
  10. 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Drivingcited by 2
  11. 2026Heterogeneous Self-Play for Realistic Highway Traffic Simulationcited by 2
  12. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
9 more
  1. 2026Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulationcited by 1
  2. 2026TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scalecited by 1
  3. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  4. 2026What Emerges and What Breaks in Self-Play Drivingcited by 1
  5. 2026CounterPlay: Counterfactual Post-Training for Self-Play Driving Policiescited by 1
  6. 2025Oryx: a Scalable Sequence Model for Many-Agent Coordination in Offline MARLcited by 0
  7. 2026General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Mergingcited by 0
  8. 2026What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planningcited by 0
  9. 2026Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policiescited by 0