mlsanityData to 7 Oct 2026

One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation

arXiv 2609.04921 ↗First posted 4 Sep 2026cs.CV cs.AI cs.LGcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Diffusion probabilistic models can capture the multi-modal, interaction-rich distribution of joint future trajectories in driving scenes. We show that a single pretrained diffusion traffic model can serve two complementary roles in the autonomous driving development loop: as an ego motion planner, and as a controllable generator of safety-critical scenarios for stress-testing the planners. On the planning side, we introduce a Single-Stream Dual-Stream (SSDS) diffusion-transformer decoder that fuses scene context via joint attention rather than late cross-attention, improving closed-loop performance on nuPlan. We further propose Decoupled Annealing Posterior Sampling with Energy (DAPSE), a training-free guidance scheme that injects arbitrary energy functions at the clean-sample level, avoiding the first-order approximation errors while requiring no auxiliary networks. Beyond planning, we leverage the same diffusion model as a controllable scenario generator to create realistic long-tail driving interactions for closed-loop evaluation. Through inference-time guidance, selected agents are steered toward safety-critical behaviors, including aggressive cut-ins, lead-vehicle braking, and combined longitudinal-lateral interactions, while preserving realistic traffic behaviors. Evaluated in closed-loop nuPlan simulations with independent black-box planners, the generated scenarios expose failure modes that remain hidden under standard benchmarks. Although the SSDS-based planner achieves stronger nominal performance, it experiences larger degradation under these challenging scenarios, demonstrating that benchmark superiority does not necessarily translate to robustness. These results demonstrate that a single learned traffic prior can simultaneously improve motion planning and provide a realistic framework for systematic planner robustness evaluation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

37 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  2. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct
  3. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries31 Dec
  4. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning25 Apr
  5. 2022Guided Conditional Diffusion for Controllable Traffic Simulation31 Oct

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  9. 2022Planning-oriented Autonomous Drivingcited by 1,068
  10. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  11. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  12. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
20 more
  1. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  2. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  3. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  4. 2024Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealingcited by 157
  5. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  6. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  7. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  8. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  9. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
  10. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
  11. 2023Language Conditioned Traffic Generationcited by 86
  12. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  13. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  14. 2025Robust Autonomy Emerges from Self-Playcited by 64
  15. 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
  16. 2024Promptable Closed-loop Traffic Simulationcited by 20
  17. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  18. 2022RAP: Risk-Aware Prediction for Robust Planningcited by 12
  19. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
  20. 2020Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energycited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation
  2. 2026Bridging Predictive Uncertainty and Safe Action: Sample-Conditioned Differentiable Planning for Autonomous Driving
  3. 2026Diffusion Forcing Planner: History-Annealed Planning with Time-Dependent Guidance for Autonomous Driving
  4. 2026G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
  5. 2025Controllable Latent Diffusion for Traffic Simulation
  6. 2026DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space
  7. 2025CoPlanner: An Interactive Motion Planner with Contingency-Aware Diffusion for Autonomous Driving
  8. 2026Temporally Decoupled Diffusion Planning for Autonomous Driving