mlsanityData to 7 Oct 2026

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

arXiv 2307.07162 ↗First posted 14 Jul 2023cs.RO cs.CLcited by 158

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 1 more
Abstract (arXiv)

In this paper, we explore the potential of using a large language model (LLM) to understand the driving environment in a human-like manner and analyze its ability to reason, interpret, and memorize when facing complex scenarios. We argue that traditional optimization-based and modular autonomous driving (AD) systems face inherent performance limitations when dealing with long-tail corner cases. To address this problem, we propose that an ideal AD system should drive like a human, accumulating experience through continuous driving and using common sense to solve problems. To achieve this goal, we identify three key abilities necessary for an AD system: reasoning, interpretation, and memorization. We demonstrate the feasibility of employing an LLM in driving scenarios by building a closed-loop system to showcase its comprehension and environment-interaction abilities. Our extensive experiments show that the LLM exhibits the impressive ability to reason and solve long-tailed cases, providing valuable insights for the development of human-like autonomous driving. The related code are available at https://github.com/PJLab-ADG/DriveLikeAHuman .

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 34 of its 50 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

34 of 50 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  10. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
22 more
  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  3. 2022Planning-oriented Autonomous Drivingcited by 1,068
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  6. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  7. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  8. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  9. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  10. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  11. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  12. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  13. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115
  14. 2023LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusioncited by 80
  15. 2023Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveyscited by 75
  16. 2023ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Modelscited by 63
  17. 2020An Exploration of Embodied Visual Explorationcited by 56
  18. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  19. 2023Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviorscited by 25
  20. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  21. 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planningcited by 9
  22. 2017An Integrated System for Perception-Driven Autonomy with Modular Robotscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 65; 2025: 45; 2026: 24.

158 papers.

Sort:

By area:

  1. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  2. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  3. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  4. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  5. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  7. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  8. 2024A Survey on 3D Gaussian Splattingcited by 221
  9. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 206
  10. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
  11. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  12. 2023A Language Agent for Autonomous Drivingcited by 161
38 more
  1. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  2. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  3. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  4. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  5. 2023Dolphins: Multimodal Language Model for Drivingcited by 124
  6. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 122
  7. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 120
  8. 2024Large Multimodal Agents: A Surveycited by 99
  9. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  10. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  11. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  12. 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 79
  13. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  14. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  15. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  16. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  17. 2023Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorldcited by 65
  18. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  19. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  20. 2023Personalized Autonomous Driving with Large Language Models: Field Experimentscited by 46
  21. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  22. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  23. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  24. 2024TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenescited by 40
  25. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learningcited by 40
  26. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  27. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  28. 2024Asynchronous Large Language Model Enhanced Planner for Autonomous Drivingcited by 38
  29. 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspectivecited by 34
  30. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 34
  31. 2024LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Drivingcited by 32
  32. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  33. 2023NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotationscited by 31
  34. 2024LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environmentscited by 31
  35. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  36. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  37. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  38. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24

50 most cited of 158 All 158 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  2. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles
  3. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles
  4. 2026A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms
  5. 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
  6. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances
  7. 2024Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving
  8. 2026Large Language Model based Interactive Decision-Making for Autonomous Driving