mlsanityData to 7 Oct 2026

DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving

arXiv 2403.16996 ↗First posted 25 Mar 2024cs.CV cs.ROcited by 67

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 1 more
Abstract (arXiv)

End-to-end driving has made significant progress in recent years, demonstrating benefits such as system simplicity and competitive driving performance under both open-loop and closed-loop settings. Nevertheless, the lack of interpretability and controllability in its driving decisions hinders real-world deployment for end-to-end driving systems. In this paper, we collect a comprehensive end-to-end driving dataset named DriveCoT, leveraging the CARLA simulator. It contains sensor data, control decisions, and chain-of-thought labels to indicate the reasoning process. We utilize the challenging driving scenarios from the CARLA leaderboard 2.0, which involve high-speed driving and lane-changing, and propose a rule-based expert policy to control the vehicle and generate ground truth labels for its reasoning process across different driving aspects and the final decisions. This dataset can serve as an open-loop end-to-end driving benchmark, enabling the evaluation of accuracy in various chain-of-thought aspects and the final decision. In addition, we propose a baseline model called DriveCoT-Agent, trained on our dataset, to generate chain-of-thought predictions and final decisions. The trained model exhibits strong performance in both open-loop and closed-loop evaluations, demonstrating the effectiveness of our proposed dataset.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 33 of its 42 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

33 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 32

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  5. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  6. 2021Video Swin Transformercited by 1,090
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  8. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  9. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  10. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  11. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  12. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
20 more
  1. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  2. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  4. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  5. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  6. 2016Learning to Navigate in Complex Environmentscited by 192
  7. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  8. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  9. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  11. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  12. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  13. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 125
  14. 2023Language Prompt for Autonomous Drivingcited by 110
  15. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  16. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  17. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  18. 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
  19. 2021Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Predictioncited by 10
  20. 2019Improved Reinforcement Learning through Imitation Learning Pretraining Towards Image-based Autonomous Drivingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 28; 2026: 28.

67 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  2. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  3. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  4. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  5. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  6. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  7. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  8. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  9. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  10. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  11. 2025AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Drivingcited by 35
  12. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
38 more
  1. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Drivingcited by 26
  2. 2025SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Drivingcited by 21
  3. 2024FASIONAD : FAst and Slow FusION Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedbackcited by 18
  4. 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 18
  5. 2025Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Modelscited by 17
  6. 2025An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraintcited by 16
  7. 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agentscited by 15
  8. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
  9. 2025Latent Chain-of-Thought World Modeling for End-to-End Drivingcited by 14
  10. 2025LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinkingcited by 13
  11. 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospectscited by 12
  12. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trendscited by 10
  13. 2024World knowledge-enhanced Reasoning Using Instruction-guided Interactor in Autonomous Drivingcited by 9
  14. 2025X-Driver: Explainable Autonomous Driving with Vision-Language Modelscited by 9
  15. 2025MTRDrive: Memory-Tool Synergistic Reasoning for Robust Autonomous Driving in Corner Casescited by 9
  16. 2025Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Drivingcited by 9
  17. 2024Generating and Evolving Reward Functions for Highway Driving with Large Language Modelscited by 8
  18. 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
  19. 2025ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Drivingcited by 7
  20. 2025VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-grounded Autonomous Drivingcited by 6
  21. 2026Accelerating Structured Chain-of-Thought in Autonomous Vehiclescited by 6
  22. 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
  23. 2026HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 5
  24. 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Drivingcited by 5
  25. 2026FROST-Drive: Scalable and Efficient End-to-End Driving with a Frozen Vision Encodercited by 4
  26. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  27. 2025PRIMEDrive-CoT: A Precognitive Chain-of-Thought Framework for Uncertainty-Aware Object Interaction in Driving Scene Scenariocited by 3
  28. 2025More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Modelscited by 3
  29. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  30. 2026Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Drivingcited by 3
  31. 2025LawFlow: Collecting and Simulating Lawyers' Thought Processes on Business Formation Case Studiescited by 2
  32. 2026OmniV2X: A Generative Foundation Planner for Efficient End-to-End Cooperative Drivingcited by 2
  33. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  34. 2026Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusioncited by 1
  35. 2026XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environmentscited by 1
  36. 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillationcited by 1
  37. 2026Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAscited by 1
  38. 2024A Novel MLLM-based Approach for Autonomous Driving in Different Weather Conditionscited by 0

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
  2. 2026ThinkDrive: Chain-of-Thought Guided Progressive Reinforcement Learning Fine-Tuning for Autonomous Driving
  3. 2026DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving
  4. 2025OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic
  5. 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
  6. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving
  7. 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving
  8. 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution