Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Models
On the map: Vision-language-action models → Failure recovery and robustness
Abstract (arXiv)
Vision-Language-Action (VLA) models excel in robotic manipulation but suffer catastrophic performance drops when canonical instructions are simply paraphrased. Although this brittleness is typically addressed through costly data scaling, our probing reveals that the root cause is architectural rather than a lack of semantic understanding. Specifically, we demonstrate that current VLAs successfully retain the correct task identity internally. The failure actually stems from the joint encoding of dynamic visual observations and text, which introduces systematic feature shifts. Because the downstream action policy is highly vulnerable to these variations, it fails to translate the preserved semantics into correct control commands. To resolve this structural bottleneck, we propose Grounded Semantic Re-binding (GSR), an elegant intervention that bypasses unstable joint routing by explicitly fusing independently extracted task semantics with native visual features to train a completely re-initialized action expert from scratch. This targeted intervention dramatically restores paraphrastic invariance using only canonical demonstrations. On the LIBERO-Para benchmark, GSR improves success rates by up to 44.6 percent. It enables lightweight models to rival massively scaled baselines and pushes state-of-the-art models to a new record PRIDE score of 70.4, outperforming the recently introduced large-scale pretrained model Xiaomi-Robotics-0 in instruction generation capabilities. Building on these insights, we also introduce ParaVLA, a natively decoupled 0.33B-parameter model exhibiting near-perfect robustness to instruction rewording. Ultimately, our work proves that robust semantic grounding can be achieved through elegant structural design, bypassing the inefficient brute-force data scaling paradigm.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 32 of its 40 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
32 of 40 references.
Closest ancestors · 8
Same direction, newest first
- 2026When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models10 Jun
- 2026LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models30 Mar
- 2026Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration6 Mar
- 2026Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment12 Feb
- 2026Stable Language Guidance for Vision-Language-Action Models7 Jan
- 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models15 Oct
- 2025Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Models8 Oct
- 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization4 Oct
Other references · 24
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2022Interactive Language: Talking to Robots in Real Timecited by 217
- 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
- 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
- 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
12 more
- 2023Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Controlcited by 32
- 2025CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Modelscited by 28
- 2025Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verificationcited by 25
- 2024Semantically Controllable Augmentations for Generalizable Robot Learningcited by 17
- 2025Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrationscited by 15
- 2026Observing and Controlling Features in Vision-Language-Action Modelscited by 14
- 2026Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Modelscited by 14
- 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 14
- 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Followingcited by 13
- 2025ROSA: Harnessing Robot States for Vision-Language and Action Alignmentcited by 5
- 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steeringcited by 4
- 2026RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential Guidance
- 2026V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
- 2026M2-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills
- 2026AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
- 2026Semantic Anchoring for Robotic Action Representations
- 2026UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models
- 2026Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning
- 2025Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations