mlsanityData to 7 Oct 2026

Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization

arXiv 2510.25616 ↗First posted 29 Oct 2025cs.LG cs.AI cs.ROcited by 34

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

The growing success of Vision-Language-Action (VLA) models stems from the promise that pretrained Vision-Language Models (VLMs) can endow agents with transferable world knowledge and vision-language (VL) grounding, laying a foundation for action models with broader generalization. Yet when these VLMs are adapted to the action modality, it remains unclear to what extent their original VL representations and knowledge are preserved. In this work, we conduct a systematic study of representation retention during VLA fine-tuning, showing that naive action fine-tuning leads to degradation of visual representations. To characterize and measure these effects, we probe VLA's hidden representations and analyze attention maps, further, we design a set of targeted tasks and methods that contrast VLA models with their counterpart VLMs, isolating changes in VL capabilities induced by action fine-tuning. We further evaluate a range of strategies for aligning visual representations and introduce a simple yet effective method that mitigates degradation and yields improved generalization to out-of-distribution (OOD) scenarios. Taken together, our analysis clarifies the trade-off between action fine-tuning and the degradation of VL representations and highlights practical approaches to recover inherited VL capabilities. Code is publicly available: https://blind-vla-paper.github.io

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 58 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

45 of 58 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model11 Sep
  2. 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture2 Sep
  3. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding2 Jun
  4. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better29 May
  5. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
  7. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 38

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  12. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
26 more
  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  3. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  4. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  5. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  6. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  7. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  8. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  9. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  10. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
  11. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  12. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 137
  13. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  14. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 103
  15. 2025Ovis2.5 Technical Reportcited by 103
  16. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  17. 2024Transformers need glasses! Information over-squashing in language taskscited by 87
  18. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  19. 2025Scaling Language-Free Visual Representation Learningcited by 62
  20. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  21. 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
  22. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  23. 2024Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalizationcited by 17
  24. 2024Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoningcited by 9
  25. 2024Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillationcited by 9
  26. 2025Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 32.

34 papers.

Sort:

By area:

  1. 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
  2. 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
  3. 2026Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Modelscited by 14
  4. 2026PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policiescited by 7
  5. 2025How Do VLAs Effectively Inherit from VLMs?cited by 6
  6. 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
  7. 2025MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalizationcited by 5
  8. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Modelscited by 4
  9. 2026Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptationcited by 4
  10. 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignmentcited by 4
  11. 2026EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillationcited by 3
  12. 2026Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Modelscited by 3
22 more
  1. 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Modelscited by 3
  2. 2026Decoding Task Progress from VLA Representationscited by 3
  3. 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Modelscited by 2
  4. 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Modelscited by 2
  5. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Modelscited by 2
  6. 2026Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillationcited by 1
  7. 2026Action with Visual Primitivescited by 1
  8. 2026FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuningcited by 1
  9. 2026Flatness Preserves Instruction Following in Vision-Language-Action Modelscited by 1
  10. 2026Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videoscited by 1
  11. 2026VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Modelscited by 1
  12. 2026RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agentscited by 1
  13. 2026Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Modelscited by 1
  14. 2026Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentationcited by 1
  15. 2026KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learningcited by 0
  16. 2026Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Modelscited by 0
  17. 2026Semantic Anchoring for Robotic Action Representationscited by 0
  18. 2026From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodabilitycited by 0
  19. 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulationcited by 0
  20. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  21. 2026Beyond the Leaderboard: Counterfactual Diagnosis of End-to-End and VLA Driving Policies Under Domain Shiftcited by 0
  22. 2026When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Models
  2. 2026VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models
  3. 2026Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
  4. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy
  5. 2026VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
  6. 2026Observing and Controlling Features in Vision-Language-Action Models
  7. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
  8. 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models