# Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations

arXiv: https://arxiv.org/abs/2509.11417 · HTML: https://mlsanity.com/paper/2509.11417/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 52 of its 63 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- Vision-language-action models / [Generalist VLA models and benchmarks](https://mlsanity.com/vla/generalist-vla-models-and-benchmarks/)

## Builds on (most cited first, 30 of 52)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)

## Built on by (most cited first, 25 of 25)

- [π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities](https://mlsanity.com/paper/2604.15483/) (2026)
- [Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting](https://mlsanity.com/paper/2509.22195/) (2025)
- [Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment](https://mlsanity.com/paper/2602.12281/) (2026)
- [LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer](https://mlsanity.com/paper/2602.10556/) (2026)
- [Scaling World Model for Hierarchical Manipulation Policies](https://mlsanity.com/paper/2602.10983/) (2026)
- [Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models](https://mlsanity.com/paper/2603.19183/) (2026)
- [Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations](https://mlsanity.com/paper/2511.22697/) (2025)
- [Scaling Cross-Embodiment World Models for Dexterous Manipulation](https://mlsanity.com/paper/2511.01177/) (2025)
- [VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing](https://mlsanity.com/paper/2605.30117/) (2026)
- [MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization](https://mlsanity.com/paper/2511.19878/) (2025)
- [Point Tracking Improves World Action Models](https://mlsanity.com/paper/2605.23856/) (2026)
- [How Do VLAs Effectively Inherit from VLMs?](https://mlsanity.com/paper/2511.06619/) (2025)
- [Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training](https://mlsanity.com/paper/2604.23121/) (2026)
- [Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment](https://mlsanity.com/paper/2607.13429/) (2026)
- [Decoupling the Declarative from the Procedural in Vision-Language-Action Models](https://mlsanity.com/paper/2606.21496/) (2026)
- [Flatness Preserves Instruction Following in Vision-Language-Action Models](https://mlsanity.com/paper/2606.23641/) (2026)
- [UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data](https://mlsanity.com/paper/2606.10683/) (2026)
- [CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding](https://mlsanity.com/paper/2607.08974/) (2026)
- [Visual Cue Guided Video Planning for Generalizable Robot Navigation](https://mlsanity.com/paper/2609.16737/) (2026)
- [Same Scene, Different Task: Skill Alignment for Compositional Generalization in VLAs](https://mlsanity.com/paper/2610.00524/) (2026)
- [How (and How Not) to Use Data Augmentation in VLA Post-Training](https://mlsanity.com/paper/2610.05994/) (2026)
- [Do VLAs Understand and Adapt to the Objects They Handle, or Simply Replay Learned Behaviors?](https://mlsanity.com/paper/2610.06078/) (2026)
- [EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning](https://mlsanity.com/paper/2610.08726/) (2026)
- [When Listening Becomes Easier: Scrubbing Visual Cues for Shortcut-Free VLAs](https://mlsanity.com/paper/2610.10912/) (2026)
- [RARA: Robust Policy Fine-Tuning through Retrieval-Anchored Representation Alignment for Robotic Manipulation](https://mlsanity.com/paper/2610.13578/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) models finetuned from vision-language models (VLMs) hold the promise of leveraging rich pretrained representations to build generalist robots across diverse tasks and environments. However, direct fine-tuning on robot data often disrupts these representations and limits generalization. We present a framework that better preserves pretrained features while adapting them for robot manipulation. Our approach introduces three components: (i) a dual-encoder design with one frozen vision encoder to retain pretrained features and another trainable for task adaptation, (ii) a string-based action tokenizer that casts continuous actions into character sequences aligned with the model's pretraining domain, and (iii) a co-training strategy that combines robot demonstrations with vision-language datasets emphasizing spatial reasoning and affordances. Evaluations in simulation and on real robots show that our method improves robustness to visual perturbations, generalization to novel instructions and environments, and overall task success compared to baselines.