mlsanityData to 7 Oct 2026

LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models

arXiv 2510.13626 ↗First posted 15 Oct 2025cs.RO cs.CL cs.CVcited by 278

On the map: Vision-language-action models → Failure recovery and robustness

Abstract (arXiv)

Visual-Language-Action (VLA) models report impressive success rates on robotic manipulation benchmarks, yet these results may mask fundamental weaknesses in robustness. We perform a systematic vulnerability analysis by introducing controlled perturbations across seven dimensions: objects layout, camera viewpoints, robot initial states, language instructions, light conditions, background textures and sensor noise. We comprehensively analyzed multiple state-of-the-art models and revealed consistent brittleness beneath apparent competence. Our analysis exposes critical weaknesses: models exhibit extreme sensitivity to perturbation factors, including camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations. Surprisingly, models are largely insensitive to language variations, with further experiments revealing that models tend to ignore language instructions completely. Our findings challenge the assumption that high benchmark scores equate to true competency and highlight the need for evaluation practices that assess reliability under realistic variation.

Key facts. Cited by 278 papers in the map (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 27 of its 29 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

27 of 29 references.

References · 27

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  8. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  9. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  10. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  11. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  12. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
15 more
  1. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  2. 2025Unified Video Action Modelcited by 195
  3. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  4. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  5. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  6. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  7. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 129
  8. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  9. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Taskscited by 119
  10. 2025Improving Vision-Language-Action Model with Online Reinforcement Learningcited by 107
  11. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  12. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  13. 2024Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policycited by 37
  14. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  15. 2025From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Modelscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 270.

278 papers.

Sort:

By area:

  1. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  2. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  3. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  4. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  5. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  6. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  7. 2026Do World Action Models Generalize Better than VLAs? A Robustness Studycited by 35
  8. 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 33
  9. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
  10. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
  11. 2026ABot-M0.5: Unified Mobility-and-Manipulation World Action Modelcited by 28
  12. 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAscited by 27
38 more
  1. 2026StarVLA-α: Reducing Complexity in Vision-Language-Action Systemscited by 27
  2. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Modelscited by 25
  3. 2026RLDX-1 Technical Reportcited by 25
  4. 2026OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulationcited by 25
  5. 2025AVA-VLA: Improving Vision-Language-Action models with Active Visual Attentioncited by 22
  6. 2026JEPA-VLA: Video Predictive Embedding is Needed for VLA Modelscited by 22
  7. 2026HoloBrain-0 Technical Reportcited by 20
  8. 2026Light-WAM: Efficient World Action Models with State-Fusion Action Decodingcited by 19
  9. 2026RHO: Your Coding Agent is Secretly a Roboticistcited by 18
  10. 2025DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Modelscited by 17
  11. 2026LIBERO-X: Robustness Litmus for Vision-Language-Action Modelscited by 17
  12. 2026VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Modelscited by 17
  13. 2026PlayWorld: Learning Robot World Models from Autonomous Playcited by 16
  14. 2025MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agentcited by 15
  15. 2026Observing and Controlling Features in Vision-Language-Action Modelscited by 14
  16. 2026FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Modelcited by 14
  17. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 14
  18. 2026Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Modelscited by 14
  19. 2026Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Modelcited by 13
  20. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulationcited by 12
  21. 2026VLANeXt: Recipes for Building Strong VLA Modelscited by 12
  22. 2026JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modelingcited by 12
  23. 2025VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modelingcited by 11
  24. 2026ROCKET: Residual-Oriented Multi-Layer Alignment for Spatially-Aware Vision-Language-Action Modelscited by 11
  25. 2026Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibrationcited by 11
  26. 2026LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Priorcited by 11
  27. 2026Geometric Action Model for Robot Policy Learningcited by 10
  28. 2026World Action Models: A Surveycited by 10
  29. 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shiftscited by 10
  30. 2025When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Modelscited by 9
  31. 2026Feedback World Model Enables Precise Guidance of Diffusion Policycited by 9
  32. 2026MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Modelscited by 9
  33. 20264D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fieldscited by 9
  34. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibilitycited by 9
  35. 2026LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Modelscited by 8
  36. 2026A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Modelcited by 8
  37. 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Modelscited by 8
  38. 2026HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulationcited by 8

50 most cited of 278 All 278 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variations
  2. 2026When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models
  3. 2026Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks
  4. 2025Experiences from Benchmarking Vision-Language-Action Models for Robotic Manipulation
  5. 2024On the Vulnerability of LLM/VLM-Controlled Robotics
  6. 2024Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks
  7. 2026LIBERO-VPro: Benchmarking Closed-Loop Visual Robustness of Robotic Foundation Models
  8. 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models