# Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models

arXiv: https://arxiv.org/abs/2610.10526 · HTML: https://mlsanity.com/paper/2610.10526/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 10 of 10)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models](https://mlsanity.com/paper/2510.13626/) (2025)
- [LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization](https://mlsanity.com/paper/2510.03827/) (2025)
- [Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification](https://mlsanity.com/paper/2510.16281/) (2025)
- [From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models](https://mlsanity.com/paper/2506.09930/) (2025)
- [Embodied Red Teaming for Auditing Robotic Foundation Models](https://mlsanity.com/paper/2411.18676/) (2024)
- [Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering](https://mlsanity.com/paper/2606.12299/) (2026)
- [Adapting Generalist Robot Policies with Semantic Reinforcement Learning](https://mlsanity.com/paper/2606.31958/) (2026)
- [VLA Grounder: Language-Conditioning Space Optimization for Black-Box VLA Models](https://mlsanity.com/paper/2607.04517/) (2026)
- [Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy](https://mlsanity.com/paper/2609.07470/) (2026)

## Abstract (arXiv)

Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: π0.5 turns on a LIBERO stove 100% of the time for "switch on the stove" and 2% for "switch on the hot plate", and a π0 checkpoint finetuned with rephrase augmentation still shows swings of up to 61 points. We characterize this sensitivity with statistically tested single-edit swings and an oracle phrase search, which shows that phrasing alone nearly closes the 21-point gap between in-distribution and out-of-distribution tasks. We then reduce it without modifying the policy. Because the sensitivity is systematic, it can be expressed as explicit rules: we score many phrasings of a few training tasks, have a large language model distill the evidence into ten to twenty rephrasing rules, and at deployment rewrite each incoming instruction once under these rules. The rules improve the frozen π0 by 16 to 27% relative on twelve held-out tasks across adversarial, VLM-generated, and human-generated phrasings, with gains concentrated on out-of-distribution tasks. The pipeline replicates on π0.5 and LIBERO, lifting in-finetune success from 93.6% to 97.8%. The method requires no retraining and no per-step verification, and applies zero-shot to unseen tasks and instructions. Project website: https://sttawm.github.io/rephrase-before-you-act