# Fail2Progress: Learning from Real-World Robot Failures with Stein Variational Inference

arXiv: https://arxiv.org/abs/2509.01746 · HTML: https://mlsanity.com/paper/2509.01746/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2025.

## Built on by (most cited first, 8 of 8)

- [Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols](https://mlsanity.com/paper/2512.02787/) (2025)
- [Hierarchical DLO Routing with Reinforcement Learning and In-Context Vision-language Models](https://mlsanity.com/paper/2510.19268/) (2025)
- [RoboVista: Evaluating Vision Language Models for Diverse Robot Applications](https://mlsanity.com/paper/2607.04610/) (2026)
- [MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution](https://mlsanity.com/paper/2609.20056/) (2026)
- [How to Utilize Failure Demo Data?: Effective Data Selection for Imitation Learning Using Distribution Differences in Attention Mechanism](https://mlsanity.com/paper/2605.07560/) (2026)
- [Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation](https://mlsanity.com/paper/2606.03385/) (2026)
- [CARF: Contrastive Attraction-Repulsion of Failure-Guided Flow Matching](https://mlsanity.com/paper/2609.21982/) (2026)
- [OpenWAM: An Open Framework for Composable World-Action Models](https://mlsanity.com/paper/2610.07922/) (2026)

## Abstract (arXiv)

Skill effect models for long-horizon manipulation tasks are prone to failures in conditions not covered by training data distributions. Therefore, enabling robots to reason about and learn from failures is necessary. We investigate the problem of efficiently generating a dataset targeted to observed failures. After fine-tuning a skill effect model on this dataset, we evaluate the extent to which the model can recover from failures and minimize future failures. We propose Fail2Progress, an approach that leverages Stein variational inference to generate multiple simulation environments in parallel, enabling efficient data sample generation similar to observed failures. Our method is capable of handling several challenging mobile manipulation tasks, including transporting multiple objects, organizing a constrained shelf, and tabletop organization. Through large-scale simulation and real-world experiments, we demonstrate that our approach excels at learning from failures across different numbers of objects. Furthermore, we show that Fail2Progress outperforms several baselines.