# Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection

arXiv: https://arxiv.org/abs/2606.09258 · HTML: https://mlsanity.com/paper/2606.09258/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models.
- Builds on 30 of its 32 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Placed in

- Vision-language-action models / [Failure recovery and robustness](https://mlsanity.com/vla/failure-recovery-and-robustness/)

## Builds on (most cited first, 30 of 30)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2503.22020/) (2025)
- [DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge](https://mlsanity.com/paper/2507.04447/) (2025)
- [MolmoAct: Action Reasoning Models that can Reason in Space](https://mlsanity.com/paper/2508.07917/) (2025)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [Video Language Planning](https://mlsanity.com/paper/2310.10625/) (2023)
- [FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2508.18269/) (2025)
- [RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning](https://mlsanity.com/paper/2409.14674/) (2024)
- [Failure Prediction at Runtime for Generative Robot Policies](https://mlsanity.com/paper/2510.09459/) (2025)
- [Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process](https://mlsanity.com/paper/2511.01718/) (2025)
- [FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models](https://mlsanity.com/paper/2510.01642/) (2025)
- [Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts](https://mlsanity.com/paper/2409.03966/) (2024)
- [Phoenix: A Motion-based Self-Reflection Framework for Fine-grained Robotic Action Correction](https://mlsanity.com/paper/2504.14588/) (2025)
- [Incorporating Task Progress Knowledge for Subgoal Generation in Robotic Manipulation through Image Edits](https://mlsanity.com/paper/2410.11013/) (2024)
- [Efficient Robotic Policy Learning via Latent Space Backward Planning](https://mlsanity.com/paper/2505.06861/) (2025)
- [Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesis](https://mlsanity.com/paper/2603.13528/) (2026)
- [ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation](https://mlsanity.com/paper/2603.27670/) (2026)
- [See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation](https://mlsanity.com/paper/2603.09292/) (2026)
- [Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policy](https://mlsanity.com/paper/2603.01953/) (2026)

## Built on by (most cited first, 5 of 5)

- [Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models](https://mlsanity.com/paper/2608.14822/) (2026)
- [World Models for Embodied Intelligence: From Plausible to Controllable to Actionable](https://mlsanity.com/paper/2609.16697/) (2026)
- [Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability](https://mlsanity.com/paper/2609.31048/) (2026)
- [Future Anchored Verification and Online Recovery for World Action Models](https://mlsanity.com/paper/2610.06280/) (2026)
- [RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors](https://mlsanity.com/paper/2610.13516/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) policies can deviate from nominal trajectories during manipulation, even when tasks remain physically feasible. Recovering from these deviations is challenging, as they push the policy into unfamiliar state spaces where direct re-planning frequently destabilizes action sequences. We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven VLAs that leverages future visual conditioning as a recovery interface. Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation. At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal. This enables the VLA to robustly map off-trajectory observations back to a familiar future. On failure-injected LIBERO, under controlled recovery timing aligned with the injected failure, B2FF increases the average success rate of a baseline VLA from 56.3% to 74.0%, demonstrating that pre-imagined milestones can guide recovery without fine-tuning the low-level action generator.