Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving
On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection
also in 3 more
Abstract (arXiv)
End-to-End (E2E) solutions have emerged as a mainstream approach for autonomous driving systems, with Vision-Language-Action (VLA) models representing a new paradigm that leverages pre-trained multimodal knowledge from Vision-Language Models (VLMs) to interpret and interact with complex real-world environments. However, these methods remain constrained by the limitations of imitation learning, which struggles to inherently encode physical rules during training. Existing approaches often rely on complex rule-based post-refinement, employ reinforcement learning that remains largely limited to simulation, or utilize diffusion guidance that requires computationally expensive gradient calculations. To address these challenges, we introduce ReflectDrive, a novel learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion. We first discretize the two-dimensional driving space to construct an action codebook, enabling the use of pre-trained Diffusion Language Models for planning tasks through fine-tuning. Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient computation. Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors. Based on this, we apply local search methods to identify unsafe tokens and determine feasible solutions, which then serve as safe anchors for inpainting-based regeneration. Evaluated on the NAVSIM benchmark, ReflectDrive demonstrates significant advantages in safety-critical trajectory generation, offering a scalable and reliable solution for autonomous driving systems.
Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 24 Sep 2025.
Builds on
34 of 41 references.
Closest ancestors · 2
Same direction, newest first
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
Other references · 32
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2025Large Language Diffusion Modelscited by 872
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
20 more
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
- 2018Learning to Drive in a Daycited by 266
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
- 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
- 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
- 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
- 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
- 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
- 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
- 2024World Models for Autonomous Driving: An Initial Surveycited by 79
- 2025Robust Autonomy Emerges from Self-Playcited by 64
- 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 62
- 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
Built on by
Citing papers per year: 2025: 8; 2026: 16.
24 papers.
Sort:
By area:
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
- 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
- 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
- 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 18
- 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
- 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 12
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoningcited by 12
- 2026Unleashing the Potential of Diffusion Models for End-to-End Autonomous Drivingcited by 11
- 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Drivingcited by 10
- 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 10
- 2025Dichotomous Diffusion Policy Optimizationcited by 6
- 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Drivingcited by 5
12 more
- 2025E0: Enhancing Generalization and Fine-Grained Control in VLA Models via Tweedie Discrete Diffusioncited by 3
- 2026Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learningcited by 3
- 2026IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Drivingcited by 2
- 2026Post-Training in End-to-End Autonomous Drivingcited by 2
- 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Drivingcited by 1
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 1
- 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Trainingcited by 1
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 0
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challengescited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Drivingcited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving28 Sep 2026
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges4 Sep 2026
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation2 Sep 2026
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving31 Aug 2026
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs3 Aug 2026
- 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training22 Jul 2026
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving9 Jul 2026
- 2026Post-Training in End-to-End Autonomous Driving9 Jul 2026
- 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System16 Jun 2026
- 2026Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learning4 Jun 2026
- 2026IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving29 May 2026
- 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving22 May 2026
12 more
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning14 May 2026
- 2026Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving26 Feb 2026
- 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving16 Feb 2026
- 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving30 Jan 2026
- 2025Dichotomous Diffusion Policy Optimization31 Dec 2025
- 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone27 Dec 2025
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future18 Dec 2025
- 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving6 Dec 2025
- 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning4 Dec 2025
- 2025E0: Enhancing Generalization and Fine-Grained Control in VLA Models via Tweedie Discrete Diffusion26 Nov 2025
- 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling13 Oct 2025
- 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving9 Oct 2025
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
- 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 18
- 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Drivingcited by 10
- 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Drivingcited by 5
- 2025E0: Enhancing Generalization and Fine-Grained Control in VLA Models via Tweedie Discrete Diffusioncited by 3
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Drivingcited by 1
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Drivingcited by 0
- 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving28 Sep 2026
- 2026WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving9 Jul 2026
- 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving22 May 2026
- 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving16 Feb 2026
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future18 Dec 2025
- 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving6 Dec 2025
- 2025E0: Enhancing Generalization and Fine-Grained Control in VLA Models via Tweedie Discrete Diffusion26 Nov 2025
- 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
- 2026Unleashing the Potential of Diffusion Models for End-to-End Autonomous Drivingcited by 11
- 2025Dichotomous Diffusion Policy Optimizationcited by 6
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challengescited by 0
- 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges4 Sep 2026
- 2026Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving26 Feb 2026
- 2025Dichotomous Diffusion Policy Optimization31 Dec 2025
- 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling13 Oct 2025
- 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 12
- 2026Post-Training in End-to-End Autonomous Drivingcited by 2
- 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Drivingcited by 1
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0
- 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation2 Sep 2026
- 2026Post-Training in End-to-End Autonomous Driving9 Jul 2026
- 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving30 Jan 2026
- 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving9 Oct 2025
- 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 0
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0
- 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving31 Aug 2026
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs3 Aug 2026
- 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning4 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- 2025VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving
- 2026DriveMA: Driving Vision-Language-Action Models with verifiable Meta-Actions
- 2026Learning Vision-Language-Action World Models for Autonomous Driving
- 2025AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
- 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving
- 2026Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion
- 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning