mlsanityData to 7 Oct 2026

GPT-Driver: Learning to Drive with GPT

arXiv 2310.01415 ↗First posted 2 Oct 2023cs.CV cs.AI cs.CLcited by 344

Not on the map. Why?

Cited by 229 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a simple yet effective approach that can transform the OpenAI GPT-3.5 model into a reliable motion planner for autonomous vehicles. Motion planning is a core challenge in autonomous driving, aiming to plan a driving trajectory that is safe and comfortable. Existing motion planners predominantly leverage heuristic methods to forecast driving trajectories, yet these approaches demonstrate insufficient generalization capabilities in the face of novel and unseen driving scenarios. In this paper, we propose a novel approach to motion planning that capitalizes on the strong reasoning capabilities and generalization potential inherent to Large Language Models (LLMs). The fundamental insight of our approach is the reformulation of motion planning as a language modeling problem, a perspective not previously explored. Specifically, we represent the planner inputs and outputs as language tokens, and leverage the LLM to generate driving trajectories through a language description of coordinate positions. Furthermore, we propose a novel prompting-reasoning-finetuning strategy to stimulate the numerical reasoning potential of the LLM. With this strategy, the LLM can describe highly precise trajectory coordinates and also its internal decision-making process in natural language. We evaluate our approach on the large-scale nuScenes dataset, and extensive experiments substantiate the effectiveness, generalization ability, and interpretability of our GPT-based motion planner. Code is now available at https://github.com/PointsCoder/GPT-Driver.

Key facts. Cited by 344 papers in the map (data through 7 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

32 of 42 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  11. 2022Planning-oriented Autonomous Drivingcited by 1,068
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
20 more
  1. 2016End to End Learning for Self-Driving Carscited by 1,000
  2. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  3. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  4. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  5. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  6. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  7. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  10. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  11. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  12. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  13. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  14. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 158
  15. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  16. 2018Baidu Apollo EM Motion Plannercited by 99
  17. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  18. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  19. 2017Optimization-Based Autonomous Racing of 1:43 Scale RC Carscited by 47
  20. 2022Point2Seq: Detecting 3D Objects as Sequencescited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 105; 2025: 125; 2026: 89.

344 papers.

Sort:

By area:

  1. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  2. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  3. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  4. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  5. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  6. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  7. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  8. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  9. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  10. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  11. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  12. 2023A Language Agent for Autonomous Drivingcited by 161
38 more
  1. 2024Behavior Generation with Latent Actionscited by 159
  2. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  3. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  4. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  5. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  6. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  7. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  8. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  9. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  10. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 122
  11. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 120
  12. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  13. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 99
  14. 2024Large Multimodal Agents: A Surveycited by 99
  15. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  16. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  17. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  18. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 82
  19. 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 79
  20. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  21. 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 75
  22. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  23. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  24. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  25. 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 67
  26. 2024Embodied Understanding of Driving Scenarioscited by 63
  27. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  28. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Drivingcited by 55
  29. 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
  30. 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
  31. 2024A Survey for Foundation Models in Autonomous Drivingcited by 51
  32. 2024Making Large Language Models Better Planners with Reasoning-Decision Alignmentcited by 51
  33. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  34. 2023Personalized Autonomous Driving with Large Language Models: Field Experimentscited by 46
  35. 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Modelscited by 46
  36. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  37. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  38. 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Userscited by 44

50 most cited of 344 All 344 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning
  2. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving
  3. 2026MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning
  4. 2026DriveMA: Rethinking Language Interfaces in Driving VLAs with One-Step Meta-Actions
  5. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
  6. 2026AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning
  7. 2025LiloDriver: A Lifelong Learning Framework for Closed-loop Motion Planning in Long-tail Autonomous Driving Scenarios
  8. 2026HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Driving