FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving
On the map: LLM reasoning → Chain-of-thought reasoning for driving
also in 2 more
Abstract (arXiv)
Vision-language models (VLMs) have advanced scene understanding and enabled explicit reasoning in end-to-end autonomous driving. However, existing methods insufficiently integrate spatial-physical evidence into planning reasoning, while reasoning adaptation remains coarse-grained and falls short of scene-specific planning demands. Furthermore, reasoning-path optimization for higher planning quality remains largely unexplored in autonomous-driving post-training. To address these limitations, we propose FactorDrive, an end-to-end autonomous driving framework for adaptive multi-step reasoning driven by planning-critical factors (PCFs). We first perform large-scale driving-domain instruction tuning to establish foundational driving knowledge. Building on this foundation, we construct PCF-CoT, a chain-of-thought (CoT) dataset that grounds planning reasoning in trajectory-relevant spatial-physical evidence and organizes reasoning around scene-specific PCFs, enabling the composition and depth of reasoning paths to adapt to different planning demands. We further introduce Quality Search-Guided Group Relative Policy Optimization (QS-GRPO), which guides Monte Carlo Tree Search (MCTS) with trajectory-level planning rewards to discover reasoning paths with higher planning quality and uses the resulting responses to optimize the policy through GRPO, thereby improving trajectory planning performance. Extensive experiments on both open-loop (nuScenes) and closed-loop-oriented (NAVSIM) benchmarks demonstrate that FactorDrive achieves state-of-the-art planning performance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 43 of its 49 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
43 of 49 references.
Closest ancestors · 11
Same direction, newest first
- 2026AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuning30 Mar
- 2025Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning30 Dec
- 2025STRIDE-QA: Visual Question Answering Dataset for Spatiotemporal Reasoning in Urban Driving Scenes14 Aug
- 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning23 Jun
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning16 Jun
- 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling23 May
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving4 Apr
- 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision19 Dec
- 2024Making Large Language Models Better Planners with Reasoning-Decision Alignment25 Aug
- 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving25 Mar
Other references · 32
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
20 more
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
- 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
- 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 154
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
- 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
- 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
- 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
- 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
- 2026Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexitycited by 48
- 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understandingcited by 35
- 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 34
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency
- 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving
- 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- 2026Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving
- 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning
- 2025OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic
- 2026MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving