DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model
On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection
also in 2 more
Abstract (arXiv)
Vision-Language-Action driving models convert a pretrained Vision-Language Model into a driving policy, allowing them to use world knowledge and follow language guidances. However, existing VLA driving models still lack driving-oriented spatial intelligence: their policies are mainly grounded on perspective image tokens and language priors, while precise motion planning requires metric geometry, top-down scene structure, and attention to safety-critical perceptual cues. This limitation makes current models vulnerable to weak visual geometry modeling and perceptual coverage in expert demonstrations. In this paper, we present DriveStack-VLA, a framework built upon a large VLM backbone. To strengthen the spatial grounding of VLA driving, we develop dual visual modeling components. We inject a Bird-Eye-View representation into the Large Language Model decoder through a DeepStack-style connection, and propose Render-Teacher Alignment to align the perceptual focus of real images with that of rasterized images. Furthermore, to bridge the gap in multimodal trajectory selection, we introduce a head-based self-critique module that ranks sampled trajectories and conditionally refines the best one. DriveStack-VLA achieves 91.6 PDMS on NAVSIMv1, 91.0 EPDMS on NAVSIMv2 (with the human penalty filter enabled), and a driving score of 79.49 with a success rate of 56.36% on the closed-loop Bench2Drive. More visualizations are available on our project page: https://anonymous.4open.science/w/drivestack-vla/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 35 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
31 of 35 references.
Closest ancestors · 4
Same direction, newest first
- 2026AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuning30 Mar
- 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving9 Jan
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
- 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
Other references · 27
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2017Neural Discrete Representation Learningcited by 4,712
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
15 more
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
- 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 151
- 2025Pseudo-Simulation for Autonomous Drivingcited by 150
- 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 120
- 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 75
- 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 41
- 2025WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Drivingcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Teaching Vision-Language-Action Models What to See and Where to Look
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
- 2026Towards Zero-Shot Transfer Across Embodiments For Driving VLAs
- 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
- 2026Less Language, More Latents: Annotation-Efficient VLAs for Driving
- 2025Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving
- 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- 2026Unifying Language-Action Understanding and Generation for Autonomous Driving