# From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment

arXiv: https://arxiv.org/abs/2502.01828 · HTML: https://mlsanity.com/paper/2502.01828/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 30 of 37)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024)
- [Inverse Reward Design](https://mlsanity.com/paper/1711.02827/) (2017)
- [REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction](https://mlsanity.com/paper/2306.15724/) (2023)
- [Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient](https://mlsanity.com/paper/2210.06718/) (2022)
- [Behavior Generation with Latent Actions](https://mlsanity.com/paper/2403.03181/) (2024)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [iVideoGPT: Interactive VideoGPTs are Scalable World Models](https://mlsanity.com/paper/2405.15223/) (2024)
- [Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals](https://mlsanity.com/paper/2407.05996/) (2024)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance](https://mlsanity.com/paper/2410.13816/) (2024)
- [Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis](https://mlsanity.com/paper/2312.08782/) (2023)
- [Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap](https://mlsanity.com/paper/2103.03236/) (2021)
- [Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress](https://mlsanity.com/paper/2410.04640/) (2024)
- [Real-Time Anomaly Detection and Reactive Planning with Large Language Models](https://mlsanity.com/paper/2407.08735/) (2024)
- [Model-Based Runtime Monitoring with Interactive Imitation Learning](https://mlsanity.com/paper/2310.17552/) (2023)
- [Inference-Time Policy Steering through Human Interactions](https://mlsanity.com/paper/2411.16627/) (2024)
- [Inverse Reinforcement Learning without Reinforcement Learning](https://mlsanity.com/paper/2303.14623/) (2023)
- [Multi-Task Interactive Robot Fleet Learning with Visual World Models](https://mlsanity.com/paper/2410.22689/) (2024)
- [Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust](https://mlsanity.com/paper/2410.01971/) (2024)

## Built on by (most cited first, 30 of 52)

- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [AdaWorld: Learning Adaptable World Models with Latent Actions](https://mlsanity.com/paper/2503.18938/) (2025)
- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models](https://mlsanity.com/paper/2506.17811/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [DynaGuide: Steering Diffusion Polices with Active Dynamic Guidance](https://mlsanity.com/paper/2506.13922/) (2025)
- [Interactive World Simulator for Robot Policy Training and Evaluation](https://mlsanity.com/paper/2603.08546/) (2026)
- [Explain Before You Answer: A Survey on Compositional Visual Reasoning](https://mlsanity.com/paper/2508.17298/) (2025)
- [Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification](https://mlsanity.com/paper/2510.16281/) (2025)
- [Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning](https://mlsanity.com/paper/2512.24426/) (2025)
- [Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment](https://mlsanity.com/paper/2602.12281/) (2026)
- [Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control](https://mlsanity.com/paper/2602.13193/) (2026)
- [TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility](https://mlsanity.com/paper/2510.07550/) (2025)
- [Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy](https://mlsanity.com/paper/2512.11218/) (2025)
- [Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation](https://mlsanity.com/paper/2505.20897/) (2025)
- [A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search](https://mlsanity.com/paper/2506.05294/) (2025)
- [VLS: Steering Pretrained Robot Policies via Vision-Language Models](https://mlsanity.com/paper/2602.03973/) (2026)
- [Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification](https://mlsanity.com/paper/2507.11662/) (2025)
- [Foundational World Models Accurately Detect Bimanual Manipulator Failures](https://mlsanity.com/paper/2603.06987/) (2026)
- [Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning](https://mlsanity.com/paper/2604.16683/) (2026)
- [Latent Policy Steering with Embodiment-Agnostic Pretrained World Models](https://mlsanity.com/paper/2507.13340/) (2025)
- [Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculation](https://mlsanity.com/paper/2510.20812/) (2025)
- [Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion](https://mlsanity.com/paper/2511.14178/) (2025)
- [RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulation](https://mlsanity.com/paper/2603.11106/) (2026)
- [Survey of Vision-Language-Action Models for Embodied Manipulation](https://mlsanity.com/paper/2508.15201/) (2025)
- [Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control](https://mlsanity.com/paper/2506.16565/) (2025)
- [When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering](https://mlsanity.com/paper/2602.22474/) (2026)
- [APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies](https://mlsanity.com/paper/2606.12366/) (2026)
- [An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges](https://mlsanity.com/paper/2512.11362/) (2025)
- [Seeing Farther and Smarter: Value-Guided Multi-Path Reflection for VLM Policy Optimization](https://mlsanity.com/paper/2602.19372/) (2026)

## Abstract (arXiv)

While generative robot policies have demonstrated significant potential in learning complex, multimodal behaviors from demonstrations, they still exhibit diverse failures at deployment-time. Policy steering offers an elegant solution to reducing the chance of failure by using an external verifier to select from low-level actions proposed by an imperfect generative policy. Here, one might hope to use a Vision Language Model (VLM) as a verifier, leveraging its open-world reasoning capabilities. However, off-the-shelf VLMs struggle to understand the consequences of low-level robot actions as they are represented fundamentally differently than the text and images the VLM was trained on. In response, we propose FOREWARN, a novel framework to unlock the potential of VLMs as open-vocabulary verifiers for runtime policy steering. Our key idea is to decouple the VLM's burden of predicting action outcomes (foresight) from evaluation (forethought). For foresight, we leverage a latent world model to imagine future latent states given diverse low-level action plans. For forethought, we align the VLM with these predicted latent states to reason about the consequences of actions in its native representation--natural language--and effectively filter proposed plans. We validate our framework across diverse robotic manipulation tasks, demonstrating its ability to bridge representational gaps and provide robust, generalizable policy steering. Videos can be found on the project website: https://yilin-wu98.github.io/forewarn/.