One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation
On the map: Diffusion models → Diffusion and flow policies for robotics
Abstract (arXiv)
Diffusion probabilistic models can capture the multi-modal, interaction-rich distribution of joint future trajectories in driving scenes. We show that a single pretrained diffusion traffic model can serve two complementary roles in the autonomous driving development loop: as an ego motion planner, and as a controllable generator of safety-critical scenarios for stress-testing the planners. On the planning side, we introduce a Single-Stream Dual-Stream (SSDS) diffusion-transformer decoder that fuses scene context via joint attention rather than late cross-attention, improving closed-loop performance on nuPlan. We further propose Decoupled Annealing Posterior Sampling with Energy (DAPSE), a training-free guidance scheme that injects arbitrary energy functions at the clean-sample level, avoiding the first-order approximation errors while requiring no auxiliary networks. Beyond planning, we leverage the same diffusion model as a controllable scenario generator to create realistic long-tail driving interactions for closed-loop evaluation. Through inference-time guidance, selected agents are steered toward safety-critical behaviors, including aggressive cut-ins, lead-vehicle braking, and combined longitudinal-lateral interactions, while preserving realistic traffic behaviors. Evaluated in closed-loop nuPlan simulations with independent black-box planners, the generated scenarios expose failure modes that remain hidden under standard benchmarks. Although the SSDS-based planner achieves stronger nominal performance, it experiences larger degradation under these challenging scenarios, demonstrating that benchmark superiority does not necessarily translate to robustness. These results demonstrate that a single learned traffic prior can simultaneously improve motion planning and provide a realistic framework for systematic planner robustness evaluation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
37 of 45 references.
Closest ancestors · 5
Same direction, newest first
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct
- 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries31 Dec
- 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning25 Apr
- 2022Guided Conditional Diffusion for Controllable Traffic Simulation31 Oct
Other references · 32
- 2017Attention Is All You Needcited by 59,932
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
- 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
- 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
20 more
- 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2024Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealingcited by 157
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
- 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
- 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
- 2019Imitation Learning as f-Divergence Minimizationcited by 121
- 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
- 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
- 2023Language Conditioned Traffic Generationcited by 86
- 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
- 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
- 2025Robust Autonomy Emerges from Self-Playcited by 64
- 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
- 2024Promptable Closed-loop Traffic Simulationcited by 20
- 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
- 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
- 2020Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energycited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation
- 2026Bridging Predictive Uncertainty and Safe Action: Sample-Conditioned Differentiable Planning for Autonomous Driving
- 2026Diffusion Forcing Planner: History-Annealed Planning with Time-Dependent Guidance for Autonomous Driving
- 2026G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
- 2025Controllable Latent Diffusion for Traffic Simulation
- 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space
- 2025CoPlanner: An Interactive Motion Planner with Contingency-Aware Diffusion for Autonomous Driving
- 2026Temporally Decoupled Diffusion Planning for Autonomous Driving