ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
On the map: LLM reasoning → Chain-of-thought reasoning for driving
also in 1 more
Abstract (arXiv)
End-to-end autonomous driving has emerged as a promising approach to unify perception, prediction, and planning within a single framework, reducing information loss and improving adaptability. However, existing methods often rely on fixed and sparse trajectory supervision, limiting their ability to capture the hierarchical reasoning process that human drivers naturally employ. To bridge this gap, we propose ReAL-AD, a Reasoning-Augmented Learning framework that structures decision-making in autonomous driving based on the three-tier human cognitive model: Driving Strategy, Driving Decision, and Driving Operation, where Vision-Language Models (VLMs) are incorporated to enhance situational awareness and structured reasoning across these levels. Specifically, we introduce: (1) the Strategic Reasoning Injector, which formulates high-level driving strategies by interpreting complex traffic contexts from VLM-generated insights; (2) the Tactical Reasoning Integrator, which refines strategic intent into interpretable tactical choices such as lane changes, overtaking, and speed adjustments; and (3) the Hierarchical Trajectory Decoder, which progressively translates tactical decisions into precise control actions for smooth and human-like trajectory execution. Extensive evaluations show that integrating our framework improves planning accuracy and safety by over 30%, making end-to-end autonomous driving more interpretable and aligned with human-like hierarchical reasoning. The project page can be found at: hrefhttps://4dvlab.github.io/projectpage/realadtexttt4dvlab.github.io/projectpage/realad
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 30 of its 58 references that are in the map. First posted on arXiv 16 Jul 2025.
Builds on
30 of 58 references.
Closest ancestors · 3
Same direction, newest first
- 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision19 Dec
- 2024FASIONAD : FAst and Slow FusION Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedback27 Nov
- 2024OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning2 May
Other references · 27
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
- 2024Yi: Open Foundation Models by 01.AIcited by 706
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2023GPT-Driver: Learning to Drive with GPTcited by 344
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
15 more
- 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
- 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
- 2023A Language Agent for Autonomous Drivingcited by 161
- 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
- 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
- 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 129
- 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
- 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Drivingcited by 55
- 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
- 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
- 2024End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotationcited by 27
Built on by
Citing papers per year: 2025: 6; 2026: 9.
15 papers.
Sort:
By area:
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
- 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 12
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
- 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Drivingcited by 5
- 2025VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Drivingcited by 5
- 2026Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planningcited by 4
- 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbationscited by 3
- 2025OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Drivingcited by 2
- 2026UniUncer: Unified Dynamic Static Uncertainty for End to End Drivingcited by 2
- 2026Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Drivingcited by 1
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation2 Sep 2026
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs21 Jun 2026
- 2026VeriDrive: Verifiable Counterfactual Supervision for Cost-Efficient Vision-Language Planning5 Jun 2026
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving29 May 2026
- 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations18 May 2026
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning14 May 2026
- 2026Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning11 Mar 2026
- 2026UniUncer: Unified Dynamic Static Uncertainty for End to End Driving8 Mar 2026
- 2026Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Driving26 Feb 2026
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future18 Dec 2025
- 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving11 Dec 2025
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail30 Oct 2025
3 more
- 2025VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving17 Oct 2025
- 2025OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Driving24 Sep 2025
- 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving31 Aug 2025
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
- 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbationscited by 3
- 2025OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Drivingcited by 2
- 2026UniUncer: Unified Dynamic Static Uncertainty for End to End Drivingcited by 2
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation2 Sep 2026
- 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving29 May 2026
- 2026Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations18 May 2026
- 2026UniUncer: Unified Dynamic Static Uncertainty for End to End Driving8 Mar 2026
- 2025OmniScene: Attention-Augmented Multimodal 4D Scene Understanding for Autonomous Driving24 Sep 2025
- 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Drivingcited by 5
- 2026Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planningcited by 4
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMscited by 0
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs21 Jun 2026
- 2026Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning11 Mar 2026
- 2025CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving31 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2026A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- 2024DualAD: Dual-Layer Planning for Reasoning in Autonomous Driving
- 2025VERDI: VLM-Embedded Reasoning for Autonomous Driving
- 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models
- 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving
- 2024WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model
- 2025ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving