# FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models

arXiv: https://arxiv.org/abs/2510.01642 · HTML: https://mlsanity.com/paper/2510.01642/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- Vision-language-action models / [Failure recovery and robustness](https://mlsanity.com/vla/failure-recovery-and-robustness/)

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [Robotic Control via Embodied Chain-of-Thought Reasoning](https://mlsanity.com/paper/2407.08693/) (2024)
- [RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics](https://mlsanity.com/paper/2406.10721/) (2024)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [Real-Time Execution of Action Chunking Flow Policies](https://mlsanity.com/paper/2506.07339/) (2025)
- [MolmoAct: Action Reasoning Models that can Reason in Space](https://mlsanity.com/paper/2508.07917/) (2025)
- [ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning](https://mlsanity.com/paper/2507.16815/) (2025)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [Yell At Your Robot: Improving On-the-Fly from Language Corrections](https://mlsanity.com/paper/2403.12910/) (2024)
- [ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model](https://mlsanity.com/paper/2502.14420/) (2025)
- [BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models](https://mlsanity.com/paper/2506.07961/) (2025)
- [Correcting Robot Plans with Natural Language Feedback](https://mlsanity.com/paper/2204.05186/) (2022)
- [RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning](https://mlsanity.com/paper/2504.18904/) (2025)
- [RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correction](https://mlsanity.com/paper/2505.12224/) (2025)
- [Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts](https://mlsanity.com/paper/2409.03966/) (2024)
- [GroundFlow: A Plug-in Module for Temporal Reasoning on 3D Point Cloud Sequential Grounding](https://mlsanity.com/paper/2506.21188/) (2025)

## Built on by (most cited first, 30 of 35)

- [Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons](https://mlsanity.com/paper/2603.02115/) (2026)
- [TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics](https://mlsanity.com/paper/2602.19313/) (2026)
- [Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols](https://mlsanity.com/paper/2512.02787/) (2025)
- [Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms](https://mlsanity.com/paper/2604.23775/) (2026)
- [Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding](https://mlsanity.com/paper/2512.22519/) (2025)
- [Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models](https://mlsanity.com/paper/2512.01946/) (2025)
- [Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring](https://mlsanity.com/paper/2605.30834/) (2026)
- [See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation](https://mlsanity.com/paper/2603.09292/) (2026)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection](https://mlsanity.com/paper/2606.09258/) (2026)
- [FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement](https://mlsanity.com/paper/2607.01111/) (2026)
- [Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering](https://mlsanity.com/paper/2606.12299/) (2026)
- [EVE: A Generator-Verifier System for Generative Policies](https://mlsanity.com/paper/2512.21430/) (2025)
- [Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models](https://mlsanity.com/paper/2605.08434/) (2026)
- [AEGIS: A Backup Reflex for Physical AI](https://mlsanity.com/paper/2606.06660/) (2026)
- [VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models](https://mlsanity.com/paper/2606.21386/) (2026)
- [BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning](https://mlsanity.com/paper/2607.29302/) (2026)
- [Audio-Visual Intelligence in Large Foundation Models](https://mlsanity.com/paper/2605.04045/) (2026)
- [RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models](https://mlsanity.com/paper/2605.09410/) (2026)
- [ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies](https://mlsanity.com/paper/2606.09630/) (2026)
- [Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation](https://mlsanity.com/paper/2606.05660/) (2026)
- [RoboBRIDGE: A Modular Framework for Bridging Policies to Robust Real-World Robotic Agents](https://mlsanity.com/paper/2607.27881/) (2026)
- [RoCo Challenge at AAAI 2026: Benchmarking Robotic Collaborative Manipulation for Assembly Towards Industrial Automation](https://mlsanity.com/paper/2603.15469/) (2026)
- [Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation](https://mlsanity.com/paper/2606.03385/) (2026)
- [ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models](https://mlsanity.com/paper/2606.09740/) (2026)
- [FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy](https://mlsanity.com/paper/2608.16697/) (2026)
- [Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2608.16843/) (2026)
- [FailBench: How Reliable are VLMs at Judging Robot Task Success?](https://mlsanity.com/paper/2609.03611/) (2026)
- [Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models](https://mlsanity.com/paper/2609.06114/) (2026)
- [CARE: Experience-Guided Atomic Corrective Execution for Vision-Language-Action Policies](https://mlsanity.com/paper/2609.24118/) (2026)

## Abstract (arXiv)

Recent advances in robotic manipulation have integrated low-level robotic control into Vision-Language Models (VLMs), extending them into Vision-Language-Action (VLA) models. Although state-of-the-art VLAs achieve strong performance in downstream robotic applications, supported by large-scale crowd-sourced robot training data, they still inevitably encounter failures during execution. Enabling robots to reason and recover from unpredictable and abrupt failures remains a critical challenge. Existing robotic manipulation datasets, collected in either simulation or the real world, primarily provide only ground-truth trajectories, leaving robots unable to recover once failures occur. Moreover, the few datasets that address failure detection typically offer only textual explanations, which are difficult to utilize directly in VLA models. To address this gap, we introduce FailSafe, a novel failure generation and recovery system that automatically produces diverse failure cases paired with executable recovery actions. FailSafe can be easily adapted to a wide range of manipulation tasks in simulators with motion planning support, enabling scalable creation of failure-action data. To demonstrate its effectiveness, we fine-tune LLaVA-OneVision-7B (LLaVA-OV-7B) to build FailSafe-VLM. Experimental results show that FailSafe-VLM successfully helps robotic arms detect and recover from potential failures, improving the performance of three state-of-the-art VLA models (Pi-0-FAST, OpenVLA, OpenVLA-OFT) by up to 22.6% on average across several tasks in ManiSkill. Furthermore, FailSafe-VLM could generalize across different spatial configurations, camera viewpoints, object and robotic embodiments.