mlsanityData to 14 Oct 2026

A DVDrive Approach for doScenes Instructed Driving Challenge

arXiv 2606.21623 ↗First posted 19 Jun 2026cs.CV cs.AIcited by 0

On the map: End-to-end autonomous driving → Reasoning VLA and VLM driving policies

Abstract (arXiv)

Instruction-conditioned trajectory prediction is an emerging problem in autonomous driving, where a model predicts the future ego trajectory not only from visual scene context and historical motion, but also from a natural-language maneuver instruction. This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive, a vision-language-action driving agent with 3D perception, reasoning, and planning capabilities. We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes and generating a 6-second ego trajectory represented by 12 future waypoints. To improve multi-view visual grounding, we further introduce a DVPE-style divided-view perception module into the OmniDrive perception head. Instead of attending globally to all camera features, the proposed module groups query features and image tokens into divided local view spaces and performs visibility-aware cross-attention within each view. This design reduces irrelevant cross-view interference and helps the model better align language instructions with local driving-relevant visual evidence. The code is publicly available at: https://github.com/feel12348/doscenes-omnidrive.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving. Builds on 26 of its 29 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

26 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  2. 2024doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation8 Dec

Other references · 24

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  3. 2022Planning-oriented Autonomous Drivingcited by 1,087
  4. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  5. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  7. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  8. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  9. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  10. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  11. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  12. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
12 more
  1. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 142
  2. 2024T2SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Drivingcited by 35
  3. 2023Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Featurecited by 22
  4. 2025Action Quality Assessment via Hierarchical Pose-guided Multi-stage Contrastive Regressioncited by 21
  5. 2025Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 20
  6. 2023Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embeddingcited by 19
  7. 2025DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistencycited by 15
  8. 2025SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generationcited by 15
  9. 2025Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoningcited by 14
  10. 2025Towards Balanced Multi-Modal Learning in 3D Human Pose Estimationcited by 12
  11. 2024DVPE: Divided View Position Embedding for Multi-View 3D Object Detectioncited by 1
  12. 2024Improving Batch Normalization with TTA for Robust Object Detection in Self-Drivingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Natural Language Instructions for Scene-Responsive Human-in-the-Loop Motion Planning in Autonomous Driving using Vision-Language-Action Models
  2. 2026PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving
  3. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
  4. 2026ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model
  5. 2026SUV: Future Scene Understanding as Video Generation for End-to-End Driving
  6. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
  7. 2024LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving
  8. 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving