mlsanityData to 14 Oct 2026

DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

arXiv 2411.15139 ↗First posted 22 Nov 2024cs.CV cs.ROcited by 334

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recently, the diffusion model has emerged as a powerful generative technique for robotic policy learning, capable of modeling multi-mode action distributions. Leveraging its capability for end-to-end autonomous driving is a promising direction. However, the numerous denoising steps in the robotic diffusion policy and the more dynamic, open-world nature of traffic scenes pose substantial challenges for generating diverse driving actions at a real-time speed. To address these challenges, we propose a novel truncated diffusion policy that incorporates prior multi-mode anchors and truncates the diffusion schedule, enabling the model to learn denoising from anchored Gaussian distribution to the multi-mode driving action distribution. Additionally, we design an efficient cascade diffusion decoder for enhanced interaction with conditional scene context. The proposed model, DiffusionDrive, demonstrates 10× reduction in denoising steps compared to vanilla diffusion policy, delivering superior diversity and quality in just 2 steps. On the planning-oriented NAVSIM dataset, with the aligned ResNet-34 backbone, DiffusionDrive achieves 88.1 PDMS without bells and whistles, setting a new record, while running at a real-time speed of 45 FPS on an NVIDIA 4090. Qualitative results on challenging scenarios further confirm that DiffusionDrive can robustly generate diverse plausible driving actions. Code and model will be available at https://github.com/hustvl/DiffusionDrive.

Key facts. Cited by 334 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 58 of its 64 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

58 of 64 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes15 Oct
  2. 2024Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation1 Aug
  3. 2024LDP: A Local Diffusion Planner for Efficient Robot Navigation and Collision Avoidance2 Jul
  4. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations6 Mar
  5. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following9 Feb
  6. 2024DexDiffuser: Generating Dexterous Grasps with Diffusion Models5 Feb
  7. 2023DICE: Diverse Diffusion Model with Scoring for Trajectory Prediction23 Oct
  8. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration11 Oct
  9. 2023Language-Guided Traffic Simulation via Scene-Level Diffusion10 Jun
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  11. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  12. 2022Guided Conditional Diffusion for Controllable Traffic Simulation31 Oct

Other references · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  12. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
34 more
  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  2. 2022Planning-oriented Autonomous Drivingcited by 1,087
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  4. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  5. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  6. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  7. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  8. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  10. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  11. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  12. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  13. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  14. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  15. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  16. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  17. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 245
  18. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  19. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  20. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  21. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 213
  22. 2023Scene as Occupancycited by 209
  23. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  24. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
  25. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  26. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  27. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  28. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  29. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  30. 2024DiG: Scalable and Efficient Diffusion Models with Gated Linear Attentioncited by 56
  31. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  32. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  33. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 33
  34. 2024DiPPeST: Diffusion-based Path Planner for Synthesizing Trajectories Applied on Quadruped Robotscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 126; 2026: 207.

334 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  2. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
  3. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  4. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  5. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  6. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  7. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
  8. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 134
  9. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  10. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
  11. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 79
  12. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
38 more
  1. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
  2. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 67
  3. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  4. 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 58
  5. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 56
  6. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 47
  7. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 47
  8. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 45
  9. 2026Driving on Registerscited by 44
  10. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  11. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 42
  12. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 42
  13. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 41
  14. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 41
  15. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  16. 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
  17. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
  18. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  19. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
  20. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  21. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 33
  22. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 30
  23. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  24. 2025DiffAD: A Unified Diffusion Modeling Approach for Autonomous Drivingcited by 26
  25. 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 26
  26. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 25
  27. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  28. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Drivingcited by 24
  29. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
  30. 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
  31. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  32. 2025GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Drivingcited by 22
  33. 2025DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Drivingcited by 21
  34. 2025StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Drivingcited by 21
  35. 2026VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Drivingcited by 20
  36. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  37. 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 19
  38. 2026Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failurescited by 19

50 most cited of 334 All 334 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Intention-aware Denoising Diffusion Model for Trajectory Prediction
  2. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout
  3. 2026Diffusion-2BC: Hybrid Diffusion and Regression Training for Offline Behavior Cloning in Autonomous Driving
  4. 2024Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting
  5. 2025DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Driving
  6. 2023TSDiT: Traffic Scene Diffusion Models With Transformers
  7. 2026One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation
  8. 2024ADM: Accelerated Diffusion Model via Estimated Priors for Robust Motion Prediction under Uncertainties