mlsanityData to 14 Oct 2026

Learning Personalized Driving Styles via Reinforcement Learning from Human Feedback

arXiv 2503.10434 ↗First posted 13 Mar 2025cs.RO cs.CV cs.LGcited by 17

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 2 more
Abstract (arXiv)

Generating human-like and adaptive trajectories is essential for autonomous driving in dynamic environments. While generative models have shown promise in synthesizing feasible trajectories, they often fail to capture the nuanced variability of personalized driving styles due to dataset biases and distributional shifts. To address this, we introduce TrajHF, a human feedback-driven finetuning framework for generative trajectory models, designed to align motion planning with diverse driving styles. TrajHF incorporates multi-conditional denoiser and reinforcement learning with human feedback to refine multi-modal trajectory generation beyond conventional imitation learning. This enables better alignment with human driving preferences while maintaining safety and feasibility constraints. TrajHF achieves performance comparable to the state-of-the-art on NavSim benchmark. TrajHF sets a new paradigm for personalized and adaptable trajectory generation in autonomous driving.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 35 of its 43 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

35 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Driving7 Mar
  2. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout5 Dec
  3. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving22 Nov
  4. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning20 Feb

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2022Planning-oriented Autonomous Drivingcited by 1,087
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  11. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  12. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
19 more
  1. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  4. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  5. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  6. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  7. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  8. 2024Diffusion Policy Policy Optimizationcited by 280
  9. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  10. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  11. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 213
  12. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  13. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  14. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  15. 2022SocialVAE: Human Trajectory Prediction using Timewise Latentscited by 67
  16. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  17. 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
  18. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHFcited by 18
  19. 2024Reinforcement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Trafficcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 10.

17 papers.

Sort:

By area:

  1. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 67
  2. 2026Driving on Registerscited by 44
  3. 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
  4. 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modelingcited by 20
  5. 2025EvaDrive: Evolutionary Adversarial Policy Optimization for End-to-End Autonomous Drivingcited by 12
  6. 2026Test-Time Trajectory Optimization for Autonomous Drivingcited by 7
  7. 2025TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Drivingcited by 4
  8. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  9. 2026Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Drivingcited by 2
  10. 2026DriveMA: Driving Vision-Language-Action Models with verifiable Meta-Actionscited by 2
  11. 2026PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learningcited by 1
  12. 2026Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modelingcited by 1
5 more
  1. 2026DriveMA: Rethinking Language Interfaces in Driving VLAs with One-Step Meta-Actionscited by 1
  2. 2026UniTeD: Unified Temporal Diffusion for Joint Perception and Planning in Autonomous Drivingcited by 1
  3. 2025CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Drivingcited by 0
  4. 2026PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personascited by 0
  5. 2026YOUDrive: Driving Style as a Composable Direction for End-to-End Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving
  2. 2026STAGE: STyle-controllable Action GEneration for personalized autonomous driving
  3. 2025Discrete Contrastive Learning for Diffusion Policies in Autonomous Driving
  4. 2025Drive As You Like: Multi-Head Diffusion with Reinforcement Learning for Personalized Driving
  5. 2025TrajFlow: Multi-modal Motion Prediction via Flow Matching
  6. 2026Driving Like Yourself: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving
  7. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving
  8. 2025Controllable Generative Trajectory Prediction via Weak Preference Alignment