mlsanityData to 14 Oct 2026

OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model

arXiv 2503.23463 ↗First posted 30 Mar 2025cs.CVcited by 176

On the map: Vision-language-action models → VLA for autonomous driving

also in 1 more
Abstract (arXiv)

We present OpenDriveVLA, a Vision Language Action model designed for end-to-end autonomous driving, built upon open-source large language models. OpenDriveVLA generates spatially grounded driving actions by leveraging multimodal inputs, including 2D and 3D instance-aware visual representations, ego vehicle states, and language commands. To bridge the modality gap between driving visual representations and language embeddings, we introduce a hierarchical vision language alignment process, projecting both 2D and 3D structured visual tokens into a unified semantic space. Furthermore, we incorporate structured agent environment ego interaction modeling into the autoregressive decoding process, enabling the model to capture fine-grained spatial dependencies and behavior-aware dynamics critical for reliable trajectory planning. Extensive experiments on the nuScenes dataset demonstrate that OpenDriveVLA achieves state-of-the-art results across open-loop trajectory planning and driving-related question answering tasks. Qualitative analyses further illustrate its capability to follow high-level driving commands and generate trajectories under challenging scenarios, highlighting its potential for next-generation end-to-end autonomous driving.

Key facts. Cited by 176 papers in the map (data through 14 Oct 2026). Placed in VLA for autonomous driving, one of 13 directions in Vision-language-action models. Builds on 59 of its 71 references that are in the map. First posted on arXiv 30 Mar 2025.

Builds on

59 of 71 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
38 more
  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  3. 2022Planning-oriented Autonomous Drivingcited by 1,087
  4. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  5. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  6. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  8. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  9. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  10. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  11. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  12. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  13. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  14. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  15. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  16. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  17. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  18. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  19. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  20. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  21. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  22. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  23. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  24. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  25. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  26. 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 132
  27. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  28. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  29. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
  30. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 114
  31. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
  32. 2023LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understandingcited by 95
  33. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  34. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 84
  35. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 78
  36. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  37. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  38. 2024Making Large Language Models Better Planners with Reasoning-Decision Alignmentcited by 52

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 60; 2026: 116.

176 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  2. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
  3. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 134
  4. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  5. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 74
  6. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  7. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  8. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  9. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 38
  10. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 37
  11. 2025AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Drivingcited by 34
  12. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
38 more
  1. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  2. 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
  3. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  4. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Drivingcited by 28
  5. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 25
  6. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 23
  7. 2025Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoningcited by 22
  8. 2025FreezeVLA: Action-Freezing Attacks against Vision-Language-Action Modelscited by 21
  9. 2025DVGT: Driving Visual Geometry Transformercited by 21
  10. 2025FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruningcited by 20
  11. 2025GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Controlcited by 19
  12. 2026DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Drivingcited by 19
  13. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 19
  14. 2025DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Modelscited by 17
  15. 2025STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Drivingcited by 17
  16. 2025OccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervisioncited by 17
  17. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planningcited by 17
  18. 2026SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Modelcited by 17
  19. 2025Latent Chain-of-Thought World Modeling for End-to-End Drivingcited by 15
  20. 2025DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving Understanding, Perception, Prediction and Planningcited by 15
  21. 2026Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Modelscited by 15
  22. 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 14
  23. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoningcited by 14
  24. 2026AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuningcited by 14
  25. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 13
  26. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanismscited by 13
  27. 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospectscited by 12
  28. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Drivingcited by 12
  29. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 12
  30. 2025Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Drivingcited by 11
  31. 2025OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Criticcited by 11
  32. 2026SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarioscited by 11
  33. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  34. 2026Learning Vision-Language-Action World Models for Autonomous Drivingcited by 11
  35. 2026CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planningcited by 11
  36. 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Drivingcited by 10
  37. 2026FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Drivingcited by 10
  38. 2026Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Modelscited by 10

50 most cited of 176 All 176 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Driving
  2. 2026Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Driving
  3. 2026Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving
  4. 2026Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving
  5. 2026OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
  6. 2025ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
  7. 2026VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving
  8. 2025Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving