mlsanityData to 14 Oct 2026

Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving

arXiv 2609.00111 ↗First posted 31 Aug 2026cs.CVcited by 9

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

We present Qwen-Drive-1.0, an initial step towards a vision-language foundation model for autonomous driving. Qwen-Drive-1.0 retains the architecture of the pretrained vision-language model (VLM) and integrates 3D perception, visual question answering, and motion planning within a unified framework. An external bird's-eye-view (BEV) perception head jointly performs 3D object detection, semantic occupancy prediction, and BEV map segmentation. It serves as a probe of the 3D information accessible from the shared representations and provides an explicit, inspectable interface to 3D scene structure. A Planning Expert conditions on shared VLM representations to generate future ego trajectories. A staged training recipe combines driving supervision with general-purpose vision-language data to acquire driving-specific competence while helping preserve broad visual understanding and instruction-following capabilities. Experiments demonstrate strong 3D perception and driving scene understanding while largely preserving general vision-language capability. Comprehensive evaluations across open-loop, pseudo-closed-loop, and closed-loop settings further show highly competitive motion-planning performance.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 93 of its 108 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

93 of 108 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning4 Dec
  2. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving25 Nov
  3. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios30 Oct
  4. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
  5. 2025STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving6 Jun
  6. 2025HMVLM: Multistage Reasoning-Enhanced Vision-Language Model for Long-Tailed Driving Scenarios6 Jun
  7. 2025Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving13 May
  8. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  9. 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving27 Mar
  10. 2025NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models17 Mar
  11. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving29 Oct
  12. 2024Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases16 Apr
4 more
  1. 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models2 Jan
  2. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  3. 2023LingoQA: Visual Question Answering for Autonomous Driving21 Dec
  4. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 77

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  11. 2021Grounded Language-Image Pre-trainingcited by 1,156
  12. 2022Planning-oriented Autonomous Drivingcited by 1,087
38 more
  1. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  5. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  6. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  8. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  10. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  11. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  12. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  13. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  14. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  15. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  16. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  17. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  18. 2026Gemma 4 Technical Reportcited by 259
  19. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
  20. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  21. 2023Scene as Occupancycited by 209
  22. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  23. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
  24. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  25. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  26. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  27. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
  28. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 142
  29. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
  30. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 126
  31. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 117
  32. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 107
  33. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  34. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 103
  35. 20243DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splattingcited by 88
  36. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  37. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  38. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026DriveZero: End-to-End Driving Beyond Human Demonstrationscited by 3
  2. 2026OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teachercited by 1
  3. 2026Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Visioncited by 1
  4. 2026GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Drivingcited by 0
  5. 2026When the Score Becomes the Target: Rethinking Metric Validity in Autonomous Drivingcited by 0
  6. 2026RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Drivingcited by 0
  7. 2026Vision-Language-Action Autonomous Driving Agent with Language-based Memorycited by 0
  8. 2026PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Drivingcited by 0
  9. 2026DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving
  2. 2025LMAD: Integrated End-to-End Vision-Language Model for Explainable Autonomous Driving
  3. 2025Structured Labeling Enables Faster Vision-Language Models for End-to-End Autonomous Driving
  4. 2025VLM-3D:End-to-End Vision-Language Models for Open-World 3D Perception
  5. 2025VLMs Guided Interpretable Decision Making for Autonomous Driving
  6. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models
  7. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
  8. 2025Goal-Based Vision-Language Driving