Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
The growing success of Vision-Language-Action (VLA) models stems from the promise that pretrained Vision-Language Models (VLMs) can endow agents with transferable world knowledge and vision-language (VL) grounding, laying a foundation for action models with broader generalization. Yet when these VLMs are adapted to the action modality, it remains unclear to what extent their original VL representations and knowledge are preserved. In this work, we conduct a systematic study of representation retention during VLA fine-tuning, showing that naive action fine-tuning leads to degradation of visual representations. To characterize and measure these effects, we probe VLA's hidden representations and analyze attention maps, further, we design a set of targeted tasks and methods that contrast VLA models with their counterpart VLMs, isolating changes in VL capabilities induced by action fine-tuning. We further evaluate a range of strategies for aligning visual representations and introduce a simple yet effective method that mitigates degradation and yields improved generalization to out-of-distribution (OOD) scenarios. Taken together, our analysis clarifies the trade-off between action fine-tuning and the degradation of VL representations and highlights practical approaches to recover inherited VL capabilities. Code is publicly available: https://blind-vla-paper.github.io
Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 58 references that are in the map. First posted on arXiv 29 Oct 2025.
Builds on
45 of 58 references.
Closest ancestors · 7
Same direction, newest first
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model11 Sep
- 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture2 Sep
- 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding2 Jun
- 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better29 May
- 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
- 2023PaLM-E: An Embodied Multimodal Language Model6 Mar
Other references · 38
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
26 more
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2020What Makes for Good Views for Contrastive Learning?cited by 843
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
- 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
- 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
- 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
- 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 137
- 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
- 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 103
- 2025Ovis2.5 Technical Reportcited by 103
- 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
- 2024Transformers need glasses! Information over-squashing in language taskscited by 87
- 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
- 2025Scaling Language-Free Visual Representation Learningcited by 62
- 2025Training Strategies for Efficient Embodied Reasoningcited by 53
- 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
- 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
- 2024Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalizationcited by 17
- 2024Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoningcited by 9
- 2024Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillationcited by 9
- 2025Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Modelscited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 32.
34 papers.
Sort:
By area:
- 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
- 2026Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Modelscited by 14
- 2026PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policiescited by 7
- 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
- 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
- 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalizationcited by 5
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Modelscited by 4
- 2026Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptationcited by 4
- 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignmentcited by 4
- 2026EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillationcited by 3
- 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Modelscited by 3
22 more
- 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Modelscited by 3
- 2026Decoding Task Progress from VLA Representationscited by 3
- 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Modelscited by 2
- 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Modelscited by 2
- 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Modelscited by 2
- 2026Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillationcited by 1
- 2026Action with Visual Primitivescited by 1
- 2026FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuningcited by 1
- 2026Flatness Preserves Instruction Following in Vision-Language-Action Modelscited by 1
- 2026Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videoscited by 1
- 2026VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Modelscited by 1
- 2026RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agentscited by 1
- 2026Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Modelscited by 1
- 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentationcited by 1
- 2026KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learningcited by 0
- 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Modelscited by 0
- 2026Semantic Anchoring for Robotic Action Representationscited by 0
- 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodabilitycited by 0
- 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulationcited by 0
- 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
- 2026Beyond the Leaderboard: Counterfactual Diagnosis of End-to-End and VLA Driving Policies Under Domain Shiftcited by 0
- 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Modelscited by 0
- 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models30 Sep 2026
- 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentation24 Sep 2026
- 2026Beyond the Leaderboard: Counterfactual Diagnosis of End-to-End and VLA Driving Policies Under Domain Shift18 Sep 2026
- 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models5 Sep 2026
- 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation19 Aug 2026
- 2026Decoding Task Progress from VLA Representations13 Aug 2026
- 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability9 Aug 2026
- 2026Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models3 Aug 2026
- 2026RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents30 Jul 2026
- 2026Semantic Anchoring for Robotic Action Representations15 Jul 2026
- 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment15 Jul 2026
- 2026VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Models5 Jul 2026
22 more
- 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models30 Jun 2026
- 2026Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos23 Jun 2026
- 2026Flatness Preserves Instruction Following in Vision-Language-Action Models22 Jun 2026
- 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models17 Jun 2026
- 2026FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning7 Jun 2026
- 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Models5 Jun 2026
- 2026Action with Visual Primitives21 May 2026
- 2026Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation15 May 2026
- 2026Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptation8 May 2026
- 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models1 May 2026
- 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training25 Apr 2026
- 2026Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models19 Mar 2026
- 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models16 Mar 2026
- 2026EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillation10 Mar 2026
- 2026PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policies23 Feb 2026
- 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Models4 Feb 2026
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models21 Jan 2026
- 2026KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning20 Jan 2026
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers20 Jan 2026
- 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning11 Jan 2026
- 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization25 Nov 2025
- 2025How Do VLAs Effectively Inherit from VLMs?10 Nov 2025
- 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
- 2026Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Modelscited by 14
- 2026PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policiescited by 7
- 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
- 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
- 2026Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptationcited by 4
- 2026EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillationcited by 3
- 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Modelscited by 3
- 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Modelscited by 3
- 2026Decoding Task Progress from VLA Representationscited by 3
- 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Modelscited by 2
- 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Modelscited by 2
12 most cited of 25 All 25 (JSON)
- 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models30 Sep 2026
- 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models5 Sep 2026
- 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation19 Aug 2026
- 2026Decoding Task Progress from VLA Representations13 Aug 2026
- 2026Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models3 Aug 2026
- 2026RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents30 Jul 2026
- 2026VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Models5 Jul 2026
- 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models30 Jun 2026
- 2026Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos23 Jun 2026
- 2026Flatness Preserves Instruction Following in Vision-Language-Action Models22 Jun 2026
- 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models17 Jun 2026
- 2026FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning7 Jun 2026
12 newest of 25 All 25 (JSON)
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
- 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalizationcited by 5
- 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignmentcited by 4
- 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentationcited by 1
- 2026Semantic Anchoring for Robotic Action Representationscited by 0
- 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodabilitycited by 0
- 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentation24 Sep 2026
- 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability9 Aug 2026
- 2026Semantic Anchoring for Robotic Action Representations15 Jul 2026
- 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment15 Jul 2026
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers20 Jan 2026
- 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization25 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Models
- 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
- 2026Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
- 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy
- 2026VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
- 2026Observing and Controlling Features in Vision-Language-Action Models
- 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
- 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models