mlsanityData to 7 Oct 2026

S-squared-VLA: Decoupling Semantic and Spatial Streams in Vision-Language-Action Models for Autonomous Driving

arXiv 2607.13926 ↗First posted 15 Jul 2026cs.ROcited by 0

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 2 more
Abstract (arXiv)

Vision-Language Models (VLMs) have demonstrated remarkable potential for high-level reasoning in autonomous driving, yet they fundamentally struggle to generate precise, low-level control actions. This limitation is rooted in a semantic-physical gap caused by the inherent mismatch between discrete language tokens and continuous trajectory planning. While Vision-Language-Action (VLA) architectures attempt to bridge this gap by unifying perception and control into a single policy, this entanglement creates a new bottleneck. Standard VLAs experience a severe spatial representation collapse, which irreversibly degrades the fine-grained spatial and geometric priors essential for safe, boundary-aware navigation. To address this limitation, we propose the S-squared-VLA, which explicitly decouples the semantic and spatial streams in Vision-Language-Action models. The semantic stream leverages hierarchical bridging to extract multi-scale VLM features for robust intent reasoning. In parallel, an independent spatial stream bypasses the autoregressive language bottleneck, directly preserving uncompressed spatial features from the visual encoder. By integrating auxiliary perception supervision, this stream explicitly equips the model with rich spatial and geometric priors. Finally, a dual-stream planning adapter fuses high-level semantic intent with precise spatial constraints via cascaded attention mechanisms. Evaluations on the NAVSIM closed-loop benchmark show that S-squared-VLA achieves a Predictive Driver Model Score (PDMS) of 87.1, establishing a new state-of-the-art for VLA models under a purely supervised fine-tuning (SFT) setting. By mitigating the spatial representation collapse of traditional VLMs, our framework significantly outperforms baselines, achieving the highest No Collision (NC) rate of 98.4 among all evaluated methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 30 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

28 of 30 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025A Survey on Vision-Language-Action Models for Autonomous Driving30 Jun
  2. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving9 Jun
  3. 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Driving26 May
  4. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  5. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  5. 2022Planning-oriented Autonomous Drivingcited by 1,068
  6. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  7. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  8. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  9. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  10. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  11. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  12. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
11 more
  1. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
  2. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  3. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  4. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  5. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  6. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 66
  7. 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
  8. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  9. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  10. 2025Multimodal Language Models See Better When They Look Shallowercited by 22
  11. 2026Attentive multilayer fusion for vision transformerscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
  2. 2026HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Driving
  3. 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving
  4. 2026AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving
  5. 2026VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving
  6. 2026RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
  7. 2026Unifying Language-Action Understanding and Generation for Autonomous Driving
  8. 2025Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving