mlsanityData to 14 Oct 2026

PADriver: Towards Personalized Autonomous Driving

arXiv 2505.05240 ↗First posted 8 May 2025cs.CVcited by 5

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

In this paper, we propose PADriver, a novel closed-loop framework for personalized autonomous driving (PAD). Built upon Multi-modal Large Language Model (MLLM), PADriver takes streaming frames and personalized textual prompts as inputs. It autoaggressively performs scene understanding, danger level estimation and action decision. The predicted danger level reflects the risk of the potential action and provides an explicit reference for the final action, which corresponds to the preset personalized prompt. Moreover, we construct a closed-loop benchmark named PAD-Highway based on Highway-Env simulator to comprehensively evaluate the decision performance under traffic rules. The dataset contains 250 hours videos with high-quality annotation to facilitate the development of PAD behavior analysis. Experimental results on the constructed benchmark show that PADriver outperforms state-of-the-art approaches on different evaluation metrics, and enables various driving modes.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 37 of its 53 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

37 of 53 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving14 Dec
  2. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Models12 Dec
  3. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles12 Oct
  4. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving4 Oct
  5. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving3 Oct
  6. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct
  7. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models28 Sep
  8. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles19 Sep
  9. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  12. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
16 more
  1. 2022Planning-oriented Autonomous Drivingcited by 1,087
  2. 2023Instruction Tuning with GPT-4cited by 686
  3. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  4. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  5. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  6. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  7. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  8. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  9. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  11. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  12. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104
  13. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 89
  14. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  15. 2023MTD-GPT: A Multi-Task Decision-Making GPT Model for Autonomous Driving at Unsignalized Intersectionscited by 38
  16. 2023MAVERIC: A Data-Driven Approach to Personalized Autonomous Drivingcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Drivingcited by 21
  2. 2026Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Drivingcited by 5
  3. 2026Driving Like Yourself: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Drivingcited by 1
  4. 2026A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarioscited by 0
  5. 2026PersonaDrive: Controllable Trajectory Prediction with Multi-Dimensional Driving Personascited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation
  2. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model
  3. 2023Personalized Autonomous Driving with Large Language Models: Field Experiments
  4. 2026ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving
  5. 2026Large Language Model based Interactive Decision-Making for Autonomous Driving
  6. 2026STAGE: STyle-controllable Action GEneration for personalized autonomous driving
  7. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
  8. 2025CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models