mlsanityData to 7 Oct 2026

KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients

arXiv 2204.13683 ↗First posted 28 Apr 2022cs.RO cs.CV cs.LGcited by 95

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Simulators offer the possibility of safe, low-cost development of self-driving systems. However, current driving simulators exhibit na"ive behavior models for background traffic. Hand-tuned scenarios are typically added during simulation to induce safety-critical situations. An alternative approach is to adversarially perturb the background traffic trajectories. In this paper, we study this approach to safety-critical driving scenario generation using the CARLA simulator. We use a kinematic bicycle model as a proxy to the simulator's true dynamics and observe that gradients through this proxy model are sufficient for optimizing the background traffic trajectories. Based on this finding, we propose KING, which generates safety-critical driving scenarios with a 20% higher success rate than black-box optimization. By solving the scenarios generated by KING using a privileged rule-based expert algorithm, we obtain training data for an imitation learning policy. After fine-tuning on this new data, we show that the policy becomes better at avoiding collisions. Importantly, our generated data leads to reduced collisions on both held-out scenarios generated via KING as well as traditional hand-crafted scenarios, demonstrating improved robustness.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 45 of its 54 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

45 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients27 Sep
  2. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coach18 Aug

Other references · 43

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2015Spatial Transformer Networkscited by 1,423
  4. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  7. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  8. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  9. 2019Learning by Cheatingcited by 361
  10. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  11. 2019Simple Black-box Adversarial Attackscited by 249
  12. 2021End-to-end Interpretable Neural Motion Plannercited by 240
31 more
  1. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  2. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  3. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  4. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  5. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  6. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  7. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  8. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  9. 2017Playing for Benchmarkscited by 169
  10. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  11. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  12. 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 145
  13. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  14. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  15. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  16. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  17. 2021Learning to drive from a world on railscited by 114
  18. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  19. 2018Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulationcited by 76
  20. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 74
  21. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  22. 2020An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Modelscited by 67
  23. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  24. 2020Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluationcited by 54
  25. 2020MonoLayout: Amodal scene layout from a single imagecited by 49
  26. 2020Scenic: A Language for Scenario Specification and Data Generationcited by 39
  27. 2020Label Efficient Visual Abstractions for Autonomous Drivingcited by 33
  28. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32
  29. 2020Driving among Flatmobiles: Bird-Eye-View occupancy grids from a monocular camera for holistic trajectory planningcited by 24
  30. 2019Efficient Black-box Assessment of Autonomous Vehicle Safetycited by 23
  31. 2020Action-Based Representation Learning for Autonomous Drivingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 24; 2024: 24; 2025: 24; 2026: 20.

95 papers.

Sort:

By area:

  1. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  4. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  5. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  6. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  7. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  8. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  9. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  10. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Drivingcited by 84
  11. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 52
  12. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
38 more
  1. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  2. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  3. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  4. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 36
  5. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  6. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  7. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  8. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  9. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
  10. 2024AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusioncited by 20
  11. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  12. 2023Coaching a Teachable Studentcited by 17
  13. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 17
  14. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  15. 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agentscited by 15
  16. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  17. 2024AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Drivingcited by 14
  18. 2024FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversarialitycited by 13
  19. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
  20. 2025Unraveling the Effects of Synthetic Data on End-to-End Autonomous Drivingcited by 12
  21. 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 12
  22. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectoriescited by 11
  23. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  24. 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulationcited by 10
  25. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generationcited by 10
  26. 2023Learning to Drive Anywherecited by 9
  27. 2025LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critical Driving Scenarioscited by 9
  28. 2024CarFormer: Self-Driving with Learned Object-Centric Representationscited by 8
  29. 2024ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stablecited by 8
  30. 2026Test-Time Trajectory Optimization for Autonomous Drivingcited by 7
  31. 2023(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learningcited by 6
  32. 2024Robust Bird's Eye View Segmentation by Adapting DINOv2cited by 6
  33. 2023InteractionNet: Joint Planning and Prediction for Autonomous Driving with Transformerscited by 4
  34. 2023Continual Driving Policy Optimization with Closed-Loop Individualized Curriculacited by 4
  35. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  36. 2023Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Drivingcited by 3
  37. 2023Generating Transferable Adversarial Simulation Scenarios for Self-Driving via Neural Renderingcited by 3
  38. 2023Bridging Data-Driven and Knowledge-Driven Approaches for Safety-Critical Scenario Generation in Automated Vehicle Validationcited by 3

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negatives
  2. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  3. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  4. 2020Safe Trajectory Planning Using Reinforcement Learning for Self Driving
  5. 2021Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving
  6. 2022Exploring the trade off between human driving imitation and safety for traffic simulation
  7. 2022Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field
  8. 2026Safety-Critical Scenarios Emerge from Initial Scenes ↗