mlsanityData to 7 Oct 2026

Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior

arXiv 2112.05077 ↗First posted 9 Dec 2021cs.CV cs.LG cs.ROcited by 136

On the map: End-to-end autonomous driving → Adversarial scenarios and safety testing

Abstract (arXiv)

Evaluating and improving planning for autonomous vehicles requires scalable generation of long-tail traffic scenarios. To be useful, these scenarios must be realistic and challenging, but not impossible to drive through safely. In this work, we introduce STRIVE, a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions. To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE. Scenario generation is formulated as an optimization in the latent space of this traffic model, perturbing an initial real-world scene to produce trajectories that collide with a given planner. A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner. Further analysis clusters generated scenarios based on collision type. We attack two planners and show that STRIVE successfully generates realistic, challenging scenarios in both cases. We additionally "close the loop" and use these scenarios to optimize hyperparameters of a rule-based planner.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 59 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

42 of 59 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2018Group Normalizationcited by 1,514
  6. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  7. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  8. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  9. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  10. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  11. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  12. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
30 more
  1. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  2. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  3. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  4. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  5. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  6. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  7. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  8. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  9. 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
  10. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  11. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  12. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  13. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  14. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  15. 2020TPNet: Trajectory Proposal Network for Motion Predictioncited by 80
  16. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  17. 2018Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulationcited by 76
  18. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 74
  19. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  20. 2020Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generationcited by 58
  21. 2020Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluationcited by 54
  22. 2019Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehiclescited by 51
  23. 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
  24. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
  25. 2021Fooling LiDAR Perception via Adversarial Trajectory Perturbationcited by 25
  26. 2021Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehiclescited by 21
  27. 2020Likelihood-Based Diverse Sampling for Trajectory Forecastingcited by 16
  28. 2020Learning Graph Structure With A Finite-State Automaton Layercited by 10
  29. 2020Adversarial Attacks on Optimization based Plannerscited by 8
  30. 2020Emergent Road Rules In Multi-Agent Driving Environmentscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 29; 2024: 31; 2025: 36; 2026: 28.

136 papers.

Sort:

By area:

  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  2. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  3. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  4. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  5. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  6. 2023The Waymo Open Sim Agents Challengecited by 86
  7. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  8. 2023DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Drivingcited by 73
  9. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  10. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 52
  11. 2022AdvDO: Realistic Adversarial Attacks for Trajectory Predictioncited by 48
  12. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
38 more
  1. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  2. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  3. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
  4. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 36
  5. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  6. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  7. 2022V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perceptioncited by 28
  8. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learningcited by 26
  9. 2023Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023cited by 25
  10. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  11. 2024PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarioscited by 24
  12. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22
  13. 2023Reinforcement Learning with Human Feedback for Realistic Traffic Simulationcited by 20
  14. 2024AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusioncited by 20
  15. 2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 20
  16. 2024Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehiclescited by 18
  17. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 17
  18. 2022Robust Trajectory Prediction against Adversarial Attackscited by 16
  19. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  20. 2023Safety-Critical Scenario Generation Via Reinforcement Learning Based Editingcited by 16
  21. 2025LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Modelscited by 16
  22. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  23. 2024AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Drivingcited by 14
  24. 2024Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Modelcited by 14
  25. 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
  26. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
  27. 2024Robust Fast Adaptation from Adversarially Explicit Task Distribution Generationcited by 12
  28. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  29. 2025CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Modelscited by 11
  30. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  31. 2025Adv-BMT: Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generationcited by 11
  32. 2022SpOT: Spatiotemporal Modeling for 3D Object Trackingcited by 10
  33. 2023PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimationcited by 10
  34. 2023Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulationcited by 10
  35. 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulationcited by 10
  36. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generationcited by 10
  37. 2025RealEngine: Simulating Autonomous Driving in Realistic Contextcited by 10
  38. 2023Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweightingcited by 9

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026CrashDiffuser: VLM-Guided Collision Intent Reasoning for Fine-Grained Safety-Critical Traffic Scenario Generation
  2. 2025DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Models
  3. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusion
  4. 2026Human Behavior-Informed Crash Scenario Generation with Real-World Crash Priors for Autonomous Vehicle Safety Evaluation
  5. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios
  6. 2025CrashAgent: Crash Scenario Generation via Multi-modal Reasoning
  7. 2025AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework
  8. 2025Enhancing Autonomous Driving Safety with Collision Scenario Integration