# LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models

arXiv: https://arxiv.org/abs/2510.13626 · HTML: https://mlsanity.com/paper/2510.13626/

## Key facts

- Cited by 303 papers in the map (data through 14 Oct 2026).
- Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models.
- Builds on 27 of its 29 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Placed in

- Vision-language-action models / [Failure recovery and robustness](https://mlsanity.com/vla/failure-recovery-and-robustness/)

## Builds on (most cited first, 27 of 27)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025)
- [WorldVLA: Towards Autoregressive Action World Model](https://mlsanity.com/paper/2506.21539/) (2025)
- [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024)
- [Unified Video Action Model](https://mlsanity.com/paper/2503.00200/) (2025)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning](https://mlsanity.com/paper/2505.18719/) (2025)
- [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024)
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [Interactive Post-Training for Vision-Language-Action Models](https://mlsanity.com/paper/2505.17016/) (2025)
- [THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation](https://mlsanity.com/paper/2402.08191/) (2024)
- [NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks](https://mlsanity.com/paper/2504.19854/) (2025)
- [Improving Vision-Language-Action Model with Online Reinforcement Learning](https://mlsanity.com/paper/2501.16664/) (2025)
- [What Can RL Bring to VLA Generalization? An Empirical Study](https://mlsanity.com/paper/2505.19789/) (2025)
- [VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12894/) (2024)
- [Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy](https://mlsanity.com/paper/2410.01345/) (2024)
- [Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization](https://mlsanity.com/paper/2505.15660/) (2025)
- [From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models](https://mlsanity.com/paper/2506.09930/) (2025)

## Built on by (most cited first, 30 of 303)

- [VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model](https://mlsanity.com/paper/2602.10098/) (2026)
- [StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing](https://mlsanity.com/paper/2604.05014/) (2026)
- [ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning](https://mlsanity.com/paper/2602.11236/) (2026)
- [Being-H0.7: A Latent World-Action Model from Egocentric Videos](https://mlsanity.com/paper/2605.00078/) (2026)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models](https://mlsanity.com/paper/2606.17846/) (2026)
- [ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?](https://mlsanity.com/paper/2606.19531/) (2026)
- [Do World Action Models Generalize Better than VLAs? A Robustness Study](https://mlsanity.com/paper/2603.22078/) (2026)
- [ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models](https://mlsanity.com/paper/2601.11404/) (2026)
- [VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models](https://mlsanity.com/paper/2512.22539/) (2025)
- [StarVLA-α: Reducing Complexity in Vision-Language-Action Systems](https://mlsanity.com/paper/2604.11757/) (2026)
- [When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs](https://mlsanity.com/paper/2602.17659/) (2026)
- [ABot-M0.5: Unified Mobility-and-Manipulation World Action Model](https://mlsanity.com/paper/2607.00678/) (2026)
- [RLDX-1 Technical Report](https://mlsanity.com/paper/2605.03269/) (2026)
- [OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation](https://mlsanity.com/paper/2605.06481/) (2026)
- [SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models](https://mlsanity.com/paper/2511.15605/) (2025)
- [AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention](https://mlsanity.com/paper/2511.18960/) (2025)
- [JEPA-VLA: Video Predictive Embedding is Needed for VLA Models](https://mlsanity.com/paper/2602.11832/) (2026)
- [Light-WAM: Efficient World Action Models with State-Fusion Action Decoding](https://mlsanity.com/paper/2606.08242/) (2026)
- [LIBERO-X: Robustness Litmus for Vision-Language-Action Models](https://mlsanity.com/paper/2602.06556/) (2026)
- [HoloBrain-0 Technical Report](https://mlsanity.com/paper/2602.12062/) (2026)
- [RHO: Your Coding Agent is Secretly a Roboticist](https://mlsanity.com/paper/2606.16458/) (2026)
- [DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models](https://mlsanity.com/paper/2511.15669/) (2025)
- [MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent](https://mlsanity.com/paper/2511.18810/) (2025)
- [PlayWorld: Learning Robot World Models from Autonomous Play](https://mlsanity.com/paper/2603.09030/) (2026)
- [VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models](https://mlsanity.com/paper/2603.22003/) (2026)
- [Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models](https://mlsanity.com/paper/2608.04404/) (2026)
- [Observing and Controlling Features in Vision-Language-Action Models](https://mlsanity.com/paper/2603.05487/) (2026)
- [FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model](https://mlsanity.com/paper/2603.10712/) (2026)
- [Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model](https://mlsanity.com/paper/2605.14950/) (2026)

## Abstract (arXiv)

Visual-Language-Action (VLA) models report impressive success rates on robotic manipulation benchmarks, yet these results may mask fundamental weaknesses in robustness. We perform a systematic vulnerability analysis by introducing controlled perturbations across seven dimensions: objects layout, camera viewpoints, robot initial states, language instructions, light conditions, background textures and sensor noise. We comprehensively analyzed multiple state-of-the-art models and revealed consistent brittleness beneath apparent competence. Our analysis exposes critical weaknesses: models exhibit extreme sensitivity to perturbation factors, including camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations. Surprisingly, models are largely insensitive to language variations, with further experiments revealing that models tend to ignore language instructions completely. Our findings challenge the assumption that high benchmark scores equate to true competency and highlight the need for evaluation practices that assess reliability under realistic variation.