mlsanityData to 7 Oct 2026

ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving

arXiv 2605.21168 ↗First posted 20 May 2026cs.AIcited by 0

On the map: End-to-end autonomous driving → Adversarial scenarios and safety testing

Abstract (arXiv)

Safety-critical scenarios are central to evaluating autonomous driving systems, yet their rarity in naturalistic logs makes simulation-based stress testing indispensable. Most scenario generation methods treat surrounding agents as adversaries, but they either (i) induce failures without explicitly modeling vehicle-road physical limits, yielding visually extreme yet physically unsolvable crashes, or (ii) enforce physical feasibility or policy feasibility in isolation, which can over-focus on aggressive maneuvers or remain tied to a controller-dependent capability boundary. We propose ScenePilot, a feasibility-guided, boundary-driven framework that targets the boundary band: scenarios that are physically solvable in principle yet still cause the deployed autonomy stack to fail. We formulate generation as constrained multi-objective reinforcement learning, combining an RSS-derived physical-feasibility score σ with an online-learned AV-risk predictor Phi, and introduce step-level feasibility-aware shielding to keep exploration near the feasibility boundary while avoiding infeasible artifacts. Experiments on SafeBench with multiple planners show that ScenePilot yields substantially higher collision rates (+6.2 percentage points) while preserving physical validity, and that adversarial fine-tuning on these boundary-band scenarios consistently reduces downstream crash rates. The code is available at https://github.com/QiyuRuan/ScenePilot.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving. Builds on 34 of its 53 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

34 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehicles20 Aug
  2. 2024SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation16 Sep
  3. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors6 Aug
  4. 2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles20 Jun
  5. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior9 Dec

Other references · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  5. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  6. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  7. 2019Adversarial Training Can Hurt Generalizationcited by 131
  8. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 129
  9. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
  10. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  11. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  12. 2022On Adversarial Robustness of Trajectory Prediction for Autonomous Vehiclescited by 89
17 more
  1. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 74
  2. 2022"Curse of rarity" for autonomous vehiclescited by 62
  3. 2020Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluationcited by 54
  4. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  5. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
  6. 2019AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithmscited by 39
  7. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  8. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 28
  9. 2020Training Adversarial Agents to Exploit Weaknesses in Deep Control Policiescited by 24
  10. 2021Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehiclescited by 21
  11. 2023Safety-Critical Scenario Generation Via Reinforcement Learning Based Editingcited by 16
  12. 2020Fundamental Considerations around Scenario-Based Testing for Automated Drivingcited by 15
  13. 2024FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversarialitycited by 13
  14. 2024NEST: A Neuromodulated Small-world Hypergraph Trajectory Prediction Model for Autonomous Drivingcited by 12
  15. 2024Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Modelcited by 11
  16. 2024A Cognitive-Driven Trajectory Prediction Model for Autonomous Driving in Mixed Autonomy Environmentcited by 10
  17. 2025Minds on the Move: Decoding Trajectory Prediction in Autonomous Driving with Cognitive Insightscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety-Critical Scenarios Emerge from Initial Scenes ↗
  2. 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resampling
  3. 2026CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation
  4. 2026AnchorDrive: LLM Scenario Rollout with Anchor-Guided Diffusion Regeneration for Safety-Critical Scenario Generation
  5. 2024A novel framework for adaptive stress testing of autonomous vehicles in multi-lane roads
  6. 2025AuthSim: Towards Authentic and Effective Safety-critical Scenario Generation for Autonomous Driving Tests
  7. 2024Generating Critical Scenarios for Testing Automated Driving Systems
  8. 2026Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation