Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving
On the map: LLM reasoning → Chain-of-thought reasoning for driving
also in 1 more
Abstract (arXiv)
End-to-end (E2E) autonomous driving aims to learn a direct mapping from visual observations to control actions. However, these E2E models often act as black boxes and struggle with complex scenarios. To address this, recent works incorporate Vision-Language Models (VLMs) to provide explicit reasoning, enhancing both interpretability and driving robustness. These approaches typically rely on pre-generated annotations, which suffer from potentially flawed labels and require costly human labor. In this work, we propose a new framework that integrates structured reasoning and geometric precision through a teacher-student architecture. The teacher model introduces reflective reasoning, where the VLM generates logical explanations and then reflectively refines the reasoning under the supervision of ground-truth action. This enhances zero-shot generalization without intermediate labels. The student model distills the teacher's reasoning capabilities via supervised fine-tuning. We also design a separate waypoint decoder that interprets textual reasoning into continuous trajectories. Our proposed solution integrates two goals: providing explicit reasoning for interpretability and delivering robust and accurate driving performance. It leverages the synergy between these two objectives within a staged inference engine to enhance driving performance and explicitly uses the reasoning to guide driving prediction. Evaluated on Waymo benchmarks, our framework outperforms classical reasoning-based baselines in zero-shot reasoning, waypoint accuracy, and inference efficiency. Our experiments validate this design, demonstrating that the reasoning text makes a significant contribution to driving inference, resulting in around a 24% improvement in performance compared to an identical model that lacks reasoning. Our work advances reasoning-driven autonomous driving toward interpretable and deployable systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 20 of its 40 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
20 of 40 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2016End to End Learning for Self-Driving Carscited by 1,000
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
- 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
- 2023GPT-Driver: Learning to Drive with GPTcited by 344
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
7 more
- 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
- 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
- 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
- 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
- 2025Trading Inference-Time Compute for Adversarial Robustnesscited by 60
- 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agentscited by 15
- 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMscited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
- 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
- 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
- 2025CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
- 2026Teaching Vision-Language-Action Models What to See and Where to Look
- 2025VERDI: VLM-Embedded Reasoning for Autonomous Driving
- 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning
- 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving