# Unifying Language-Action Understanding and Generation for Autonomous Driving arXiv: https://arxiv.org/abs/2603.01441 · HTML: https://mlsanity.com/paper/2603.01441/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in VLA for autonomous driving, one of 13 directions in Vision-language-action models. - Builds on 54 of its 63 references that are in the map. - First posted on arXiv 2 Mar 2026. ## Placed in - Vision-language-action models / [VLA for autonomous driving](https://mlsanity.com/vla/vla-for-autonomous-driving/) - End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/) ## Builds on (most cited first, 30 of 54) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024) - [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022) - [Show-o: One Single Transformer to Unify Multimodal Understanding and Generation](https://mlsanity.com/paper/2408.12528/) (2024) - [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025) - [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023) - [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023) - [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024) - [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022) - [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023) - [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022) - [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023) - [DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2411.15139/) (2024) - [LMDrive: Closed-Loop End-to-End Driving with Large Language Models](https://mlsanity.com/paper/2312.07488/) (2023) - [VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning](https://mlsanity.com/paper/2402.13243/) (2024) - [AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.13757/) (2025) - [Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving](https://mlsanity.com/paper/2406.03877/) (2024) - [Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation](https://mlsanity.com/paper/2406.06978/) (2024) - [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022) - [SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation](https://mlsanity.com/paper/2405.19620/) (2024) - [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023) - [DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving](https://mlsanity.com/paper/2312.09245/) (2023) - [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024) - [GenAD: Generative End-to-End Autonomous Driving](https://mlsanity.com/paper/2402.11502/) (2024) - [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024) - [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025) - [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023) ## Built on by (most cited first, 9 of 9) - [EponaV2: Driving World Model with Comprehensive Future Reasoning](https://mlsanity.com/paper/2605.14696/) (2026) - [CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies](https://mlsanity.com/paper/2605.04470/) (2026) - [BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving](https://mlsanity.com/paper/2606.08684/) (2026) - [AnchorVLA: Bridging Discrete Decisions and Continuous Trajectories for Vision-Language-Action Planning](https://mlsanity.com/paper/2607.03182/) (2026) - [Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems](https://mlsanity.com/paper/2606.00090/) (2026) - [SPVC: Structured and Panoptic Video Fixing for Cross-Dataset Driving Scene Rendering](https://mlsanity.com/paper/2608.17420/) (2026) - [GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving](https://mlsanity.com/paper/2609.15169/) (2026) - [Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs](https://mlsanity.com/paper/2609.15322/) (2026) - [CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Driving](https://mlsanity.com/paper/2609.34387/) (2026) ## Abstract (arXiv) Vision-Language-Action (VLA) models are emerging as a promising paradigm for end-to-end autonomous driving, valued for their potential to leverage world knowledge and reason about complex driving scenes. However, existing methods suffer from two critical limitations: a persistent misalignment between language instructions and action outputs, and the inherent inefficiency of typical auto-regressive action generation. In this paper, we introduce LinkVLA, a novel architecture that directly addresses these challenges to enhance both alignment and efficiency. First, we establish a structural link by unifying language and action tokens into a shared discrete codebook, processed within a single multi-modal model. This structurally enforces cross-modal consistency from the ground up. Second, to create a deep semantic link, we introduce an auxiliary action understanding objective that trains the model to generate descriptive captions from trajectories, fostering a bidirectional language-action mapping. Finally, we replace the slow, step-by-step generation with a two-step coarse-to-fine generation method C2F that efficiently decodes the action sequence, saving 86% inference time. Experiments on closed-loop driving benchmarks show consistent gains in instruction following accuracy and driving performance, alongside reduced inference latency.