mlsanityData to 7 Oct 2026

When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models

arXiv 2606.11906 ↗First posted 10 Jun 2026cs.CLcited by 3

On the map: Vision-language-action models → Failure recovery and robustness

Abstract (arXiv)

Vision-Language-Action (VLA) models have shown strong performance in language-conditioned robotic manipulation, yet their robustness to linguistic variation remains poorly understood. In this work, we present the first systematic multilingual evaluation of VLA models by translating the LIBERO benchmark into ten languages, revealing severe performance degradation under non-English instructions, with success rates dropping by 30-50%. Through fine-grained analysis of task executions, we find that language influence is highly non-uniform across steps: certain steps exhibit strong language dependence and dominate overall task failure, while others are largely language-agnostic. Based on this insight, we propose a step-wise inference-time intervention that aligns representations according to step language sensitivity, substantially improving performance under linguistic variation. Our results indicate that language robustness in VLA models is fundamentally a step-wise control problem, highlighting the importance of temporally structured analysis for reliable embodied agents.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 18 of its 20 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

18 of 20 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  5. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  8. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  9. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  10. 2024PALO: A Polyglot Large Multimodal Model for 5B Peoplecited by 34
  11. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  12. 2025Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variationscited by 24
6 more
  1. 2025RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbationscited by 23
  2. 2025Improving Pre-Trained Vision-Language-Action Policies with Model-Based Searchcited by 12
  3. 2025CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Interventioncited by 11
  4. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineeringcited by 7
  5. 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languagescited by 6
  6. 2025Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Modelscited by 0
  2. 2026Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognitioncited by 0
  3. 2026Outcome-Conditioned End-Effector Geometry Across Vision-Language-Action Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models
  2. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
  3. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
  4. 2026Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration
  5. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  6. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
  7. 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering
  8. 2025Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Models