mlsanityData to 7 Oct 2026

Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion

arXiv 2602.20577 ↗First posted 24 Feb 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 2 more
Abstract (arXiv)

Large Language Models (LLMs) and Vision-Language Models (VLMs) have emerged as promising candidates for end-to-end autonomous driving. However, these models typically face challenges in inference latency, action precision, and explainability. Existing autoregressive approaches struggle with slow token-by-token generation, while prior diffusion-based planners often rely on verbose, general-purpose language tokens that lack explicit geometric structure. In this work, we propose Masked Vision-Language-Action Diffusion for Autonomous Driving (MVLAD-AD), a novel framework designed to bridge the gap between efficient planning and semantic explainability via a masked vision-language-action diffusion model. Unlike methods that force actions into the language space, we introduce a discrete action tokenization strategy that constructs a compact codebook of kinematically feasible waypoints from real-world driving distributions. Moreover, we propose geometry-aware embedding learning to ensure that embeddings in the latent space approximate physical geometric metrics. Finally, an action-priority decoding strategy is introduced to prioritize trajectory generation. Extensive experiments on nuScenes and derived benchmarks demonstrate that MVLAD-AD achieves superior efficiency and outperforms state-of-the-art autoregressive and diffusion baselines in planning precision, while providing high-fidelity and explainable reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 42 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

38 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving1 Feb
  2. 2025ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving18 Aug
  3. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving19 Dec
  4. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  5. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving14 Dec
  6. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 32

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  11. 2022Planning-oriented Autonomous Drivingcited by 1,068
  12. 2019Attention is not Explanationcited by 916
20 more
  1. 2025Large Language Diffusion Modelscited by 872
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  4. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  5. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  6. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  7. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  8. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  9. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
  10. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 106
  11. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  12. 2021Explanations in Autonomous Driving: A Surveycited by 80
  13. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  14. 2023Personalized Autonomous Driving with Large Language Models: Field Experimentscited by 46
  15. 2024TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenescited by 40
  16. 2024Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Drivingcited by 33
  17. 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
  18. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trendscited by 10
  19. 2023Textual Explanations for Automated Commentary Drivingcited by 6
  20. 2025ALN-P3: Unified Language Alignment for Perception, Prediction, and Planning in Autonomous Drivingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers
  2. 2026VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving
  3. 2026Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving
  4. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
  5. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving
  6. 2026S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving
  7. 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving
  8. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model