# Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving

arXiv: https://arxiv.org/abs/2608.30144 · HTML: https://mlsanity.com/paper/2608.30144/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 12 of its 107 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- Vision-language-action models / [VLA for autonomous driving](https://mlsanity.com/vla/vla-for-autonomous-driving/)
- End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/)

## Builds on (most cited first, 12 of 12)

- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [Visual Adversarial Attack on Vision-Language Models for Autonomous Driving](https://mlsanity.com/paper/2411.18275/) (2024)
- [Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving](https://mlsanity.com/paper/2501.13563/) (2025)
- [Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models](https://mlsanity.com/paper/2510.13237/) (2025)
- [Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms](https://mlsanity.com/paper/2604.23775/) (2026)
- [Causal Scene Narration with Runtime Safety Supervision for Vision-Language-Action Driving](https://mlsanity.com/paper/2604.01723/) (2026)
- [Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving](https://mlsanity.com/paper/2605.10034/) (2026)
- [Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarios](https://mlsanity.com/paper/2412.07518/) (2024)
- [DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving](https://mlsanity.com/paper/2603.18315/) (2026)
- [Multimodal Backdoor Attack on VLMs for Autonomous Driving via Graffiti and Cross-Lingual Triggers](https://mlsanity.com/paper/2604.04630/) (2026)
- [LLM-Generated Fault Scenarios for Evaluating Perception-Driven Lane Following in Autonomous Edge Systems](https://mlsanity.com/paper/2604.07362/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) models are reshaping autonomous driving (AD) by unifying perception, reasoning, and control through language, enabling semantic grounding, interpretable decisions, and better long-tail generalization. But language is expensive onboard: latency and memory budgets are tight, and autoregressive decoding is inherently sequential. This work reframes the central question as when and where language should act at inference, since inference cost recurs at every deployed frame while training cost is paid once. We introduce the Language Residue taxonomy to organize methods by their inference-time use of language: train-time-only supervision (L1), latent non-textual reasoning (L2), conditional invocation (L3), and full per-frame generation (L4). We review representative methods and tag each across five deployment axes (latency, parameters, memory, FLOPs, tokens), analyzing them on major open- and closed-loop driving benchmarks (e.g., nuScenes, NAVSIM, Bench2Drive). We further trace how efficient methods from NLP/LLM are adapted in AD, identifying the constraints and motivations driving these adaptations. A continuously updated repository will be available at Github.