How Do VLAs Effectively Inherit from VLMs?
On the map: Vision-language-action models → Generalist VLA models and benchmarks
Abstract (arXiv)
Vision-language-action (VLA) models hold the promise to attain generalizable embodied control. To achieve this, a pervasive paradigm is to leverage the rich vision-semantic priors of large vision-language models (VLMs). However, the fundamental question persists: How do VLAs effectively inherit the prior knowledge from VLMs? To address this critical question, we introduce a diagnostic benchmark, GrinningFace, an emoji tabletop manipulation task where the robot arm is asked to place objects onto printed emojis corresponding to language instructions. This task design is particularly revealing -- knowledge associated with emojis is ubiquitous in Internet-scale datasets used for VLM pre-training, yet emojis themselves are largely absent from standard robotics datasets. Consequently, they provide a clean proxy: successful task completion indicates effective transfer of VLM priors to embodied control. We implement this diagnostic task in both simulated environment and a real robot, and compare various promising techniques for knowledge transfer. Specifically, we investigate the effects of parameter-efficient fine-tuning, VLM freezing, co-training, predicting discretized actions, and predicting latent actions. Through systematic evaluation, our work not only demonstrates the critical importance of preserving VLM priors for the generalization of VLA but also establishes guidelines for future research in developing truly generalizable embodied AI systems.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 44 of its 51 references that are in the map. First posted on arXiv 10 Nov 2025.
Builds on
44 of 51 references.
Closest ancestors · 8
Same direction, newest first
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting26 Sep
- 2025Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations14 Sep
- 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better29 May
- 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization21 May
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalization22 Apr
- 2024π0: A Vision-Language-Action Flow Model for General Robot Control31 Oct
- 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
Other references · 36
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2022A Generalist Agentcited by 808
24 more
- 2019Solving Rubik's Cube with a Robot Handcited by 734
- 2024PaliGemma: A versatile 3B VLM for transfercited by 696
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
- 2020Reinforcement Learning with Augmented Datacited by 468
- 2024Latent Action Pretraining from Videoscited by 321
- 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
- 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 181
- 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 133
- 2023Learning to Act without Actionscited by 121
- 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 113
- 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
- 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 84
- 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 82
- 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 72
- 2020RetinaGAN: An Object-aware Approach to Sim-to-Real Transfercited by 70
- 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
- 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
- 2025Don't Blind Your VLA: Aligning Visual Representations for OOD Generalizationcited by 35
- 2025Robix: A Unified Model for Robot Interaction, Reasoning and Planningcited by 23
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 8
- 2026Learning Additively Compositional Latent Actions for Embodied AIcited by 7
- 2026Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-Systemcited by 6
- 2026Grounded World Model for Semantically Generalizable Planningcited by 1
- 2026RoboSemanticBench: Diagnosing Semantic Grounding in Action Prediction for VLA Modelscited by 1
- 2026RoboSemanticBench: Diagnosing Semantic Grounding in Action Prediction for VLA Models1 Jun 2026
- 2026Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System27 Apr 2026
- 2026Grounded World Model for Semantically Generalizable Planning13 Apr 2026
- 2026Learning Additively Compositional Latent Actions for Embodied AI3 Apr 2026
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models6 Mar 2026
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers20 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025IA-VLA: Input Augmentation for Vision-Language-Action models in settings with semantically complex tasks
- 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
- 2024A Survey on Vision-Language-Action Models for Embodied AI
- 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models
- 2024What Matters in Building Vision-Language-Action Models for Generalist Robots
- 2025Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions
- 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
- 2026Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data