mlsanityData to 7 Oct 2026

Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning

arXiv 2410.05582 ↗First posted 8 Oct 2024cs.ROcited by 41

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Autonomous driving necessitates the ability to reason about future interactions between traffic agents and to make informed evaluations for planning. This paper introduces the textitGen-Drive framework, which shifts from the traditional prediction and deterministic planning framework to a generation-then-evaluation planning paradigm. The framework employs a behavior diffusion model as a scene generator to produce diverse possible future scenarios, thereby enhancing the capability for joint interaction reasoning. To facilitate decision-making, we propose a scene evaluator (reward) model, trained with pairwise preference data collected through VLM assistance, thereby reducing human workload and enhancing scalability. Furthermore, we utilize an RL fine-tuning framework to improve the generation quality of the diffusion model, rendering it more effective for planning tasks. We conduct training and closed-loop planning tests on the nuPlan dataset, and the results demonstrate that employing such a generation-then-evaluation strategy outperforms other learning-based approaches. Additionally, the fine-tuned generative driving policy shows significant enhancements in planning performance. We further demonstrate that utilizing our learned reward model for evaluation or RL fine-tuning leads to better planning performance compared to relying on human-designed rewards. Project website: https://mczhi.github.io/GenDrive.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

34 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review18 Jul
  2. 2024Large-scale Reinforcement Learning for Diffusion Models20 Jan
  3. 2023SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Models27 Nov
  4. 2023A Diffusion-Model of Joint Interactive Navigation21 Sep
  5. 2023Language-Guided Traffic Simulation via Scene-Level Diffusion10 Jun
  6. 2023Training Diffusion Models with Reinforcement Learning22 May
  7. 2022Guided Conditional Diffusion for Controllable Traffic Simulation31 Oct

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  5. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  6. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  7. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  8. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  9. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  10. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  11. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  12. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
15 more
  1. 2022Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Drivingcited by 71
  2. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  3. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  4. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  5. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  6. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
  7. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
  8. 2020MATS: An Interpretable Trajectory Forecasting Representation for Planning and Controlcited by 30
  9. 2022Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedbackcited by 26
  10. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  11. 2023Reinforcement Learning with Human Feedback for Realistic Traffic Simulationcited by 20
  12. 2023What Truly Matters in Trajectory Prediction for Autonomous Driving?cited by 17
  13. 2023Interactive Joint Planning for Autonomous Vehiclescited by 13
  14. 2023Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latentcited by 10
  15. 2024Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Drivingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 19; 2026: 20.

41 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  2. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  3. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
  4. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  5. 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
  6. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 31
  7. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  8. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  9. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 19
  10. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  11. 2026AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuningcited by 13
  12. 2024CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Modelcited by 12
29 more
  1. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  2. 2025RDPO: Real Data Preference Optimization for Physics Consistency Video Generationcited by 8
  3. 2025TrajDiff: End-to-end Autonomous Driving without Perception Annotationcited by 8
  4. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
  5. 2025Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Drivingcited by 7
  6. 2026Generative Scenario Rollouts for End-to-End Autonomous Drivingcited by 6
  7. 2026MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systemscited by 4
  8. 2025Diffusion-based Planning with Learned Viability Filterscited by 2
  9. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  10. 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generationcited by 2
  11. 2025DriveMRP: Enhancing Vision-Language Models with Synthetic Motion Data for Motion Risk Predictioncited by 2
  12. 2026CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Drivingcited by 2
  13. 2026SCORP: Scene-Consistent Multi-agent Diffusion Planning with Stable Online Reinforcement Post-Training for Cooperative Drivingcited by 2
  14. 2026FeaXDrive: Feasibility-aware Trajectory-Centric Diffusion Planning for End-to-End Autonomous Drivingcited by 2
  15. 2026CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Drivingcited by 2
  16. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  17. 2026A Unified Candidate Set with Scene-Adaptive Refinement via Diffusion for End-to-End Autonomous Drivingcited by 1
  18. 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Modelcited by 1
  19. 2026MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Drivingcited by 1
  20. 2026DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Drivingcited by 1
  21. 2026Motion Planning in Compressed Representation Spacescited by 1
  22. 2026Teaching Vision-Language-Action Models What to See and Where to Lookcited by 1
  23. 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
  24. 2025VLM as Strategist: Adaptive Generation of Safety-critical Testing Scenarios via Guided Diffusioncited by 0
  25. 2026Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modelingcited by 0
  26. 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 0
  27. 2026One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulationcited by 0
  28. 2026GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Drivingcited by 0
  29. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimization-Guided Diffusion for Interactive Scene Generation
  2. 2024Versatile Behavior Diffusion for Generalized Traffic Agent Simulation
  3. 2025EvaDrive: Evolutionary Adversarial Policy Optimization for End-to-End Autonomous Driving
  4. 2025Drive As You Like: Multi-Head Diffusion with Reinforcement Learning for Personalized Driving
  5. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving
  6. 2025RealDrive: Retrieval-Augmented Driving with Diffusion Models
  7. 2026HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving
  8. 2025Generative Models in Decision Making: A Survey