mlsanityData to 14 Oct 2026

DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving

arXiv 2509.17940 ↗First posted 22 Sep 2025cs.RO cs.CVcited by 39

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 1 more
Abstract (arXiv)

End-to-end autonomous driving has substantially progressed by directly predicting future trajectories from raw perception inputs, which bypasses traditional modular pipelines. However, mainstream methods trained via imitation learning suffer from critical safety limitations, as they fail to distinguish between trajectories that appear human-like but are potentially unsafe. Some recent approaches attempt to address this by regressing multiple rule-driven scores but decoupling supervision from policy optimization, resulting in suboptimal performance. To tackle these challenges, we propose DriveDPO, a Safety Direct Preference Optimization Policy Learning framework. First, we distill a unified policy distribution from human imitation similarity and rule-based safety scores for direct policy optimization. Further, we introduce an iterative Direct Preference Optimization stage formulated as trajectory-level preference alignment. Extensive experiments on the NAVSIM benchmark demonstrate that DriveDPO achieves a new state-of-the-art PDMS of 90.0. Furthermore, qualitative results across diverse challenging scenarios highlight DriveDPO's ability to produce safer and more reliable driving behaviors.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 51 of its 59 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

51 of 59 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Learning Personalized Driving Styles via Reinforcement Learning from Human Feedback13 Mar
  2. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Driving7 Mar
  3. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving22 Nov
  4. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning8 Oct
  5. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning20 Feb
  6. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec

Other references · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  8. 2022Planning-oriented Autonomous Drivingcited by 1,087
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  10. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  11. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  12. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
33 more
  1. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  2. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  4. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  5. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  6. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  7. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
  8. 2018Learning to Drive in a Daycited by 273
  9. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  10. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  11. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  12. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  13. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  14. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
  15. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
  16. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 153
  17. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
  18. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  19. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  20. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  21. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 97
  22. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  23. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
  24. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  25. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  26. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
  27. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  28. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 42
  29. 2025Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigationcited by 37
  30. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safetycited by 37
  31. 2024Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Levelcited by 22
  32. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  33. 2023Reward Finetuning for Faster and More Accurate Unsupervised Object Discoverycited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 35.

39 papers.

Sort:

By area:

  1. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 30
  2. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  3. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  4. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 19
  5. 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Modelscited by 14
  6. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  7. 2026SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse Worldcited by 8
  8. 2026RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Frameworkcited by 8
  9. 2026The DAWN of World-Action Interactive Modelscited by 8
  10. 2026Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negativescited by 8
  11. 2026Do Open-Loop Metrics Predict Closed-Loop Driving? A Cross-Benchmark Correlation Study of NAVSIM and Bench2Drivecited by 6
  12. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Drivingcited by 6
27 more
  1. 2026CausalDrive: Real-time Causal World Models for Autonomous Drivingcited by 5
  2. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  3. 2026Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Drivingcited by 4
  4. 2026RISE: Adaptive Imagination for World Action Modelscited by 4
  5. 2026Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Drivingcited by 3
  6. 2026VLGA: Vision-Language-Geometry-Action Models for Autonomous Drivingcited by 3
  7. 2026SafeAlign-VLA: A Negative-Enhanced Safe Alignment Framework for Risk-Aware Autonomous Drivingcited by 2
  8. 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Drivingcited by 2
  9. 2026Post-Training in End-to-End Autonomous Drivingcited by 2
  10. 2026A Unified Candidate Set with Scene-Adaptive Refinement via Diffusion for End-to-End Autonomous Drivingcited by 1
  11. 2026Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Drivingcited by 1
  12. 2026Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modelingcited by 1
  13. 2026DOG-DPO:Dynamic Optimization in Geometry for Safety Alignmentcited by 1
  14. 2026UniTeD: Unified Temporal Diffusion for Joint Perception and Planning in Autonomous Drivingcited by 1
  15. 2026Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Drivingcited by 1
  16. 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Trainingcited by 1
  17. 2026GSDrive: Reinforcing Driving Policies by Multi-mode Future Trajectory Probing with 3D Gaussian Splatting Environmentcited by 0
  18. 2026DriveSafer: End-to-End Autonomous Driving with Safety Guidancecited by 0
  19. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Modelcited by 0
  20. 2026TPS-Drive: Task-Guided Representation Purification for VLM-based Autonomous Drivingcited by 0
  21. 2026GraphBEV++: Multi-Modal Feature Alignment for Autonomous Drivingcited by 0
  22. 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0
  23. 2026Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMscited by 0
  24. 2026FeasibleFlow: One-Step Joint Transport of Configuration Feasibility and Trajectories for End-to-End Drivingcited by 0
  25. 2026FocusDrive: Reasoning with Visual Focus for Autonomous Drivingcited by 0
  26. 2026Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Drivingcited by 0
  27. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling
  2. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  3. 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving
  4. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
  5. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving
  6. 2026DriveZero: End-to-End Driving Beyond Human Demonstrations
  7. 2020Safe Trajectory Planning Using Reinforcement Learning for Self Driving
  8. 2025TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Data