# Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation

arXiv: https://arxiv.org/abs/2502.16707 · HTML: https://mlsanity.com/paper/2502.16707/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models](https://mlsanity.com/paper/2310.10639/) (2023)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [PaLI-X: On Scaling up a Multilingual Vision and Language Model](https://mlsanity.com/paper/2305.18565/) (2023)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Physically Grounded Vision-Language Models for Robotic Manipulation](https://mlsanity.com/paper/2309.02561/) (2023)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Yell At Your Robot: Improving On-the-Fly from Language Corrections](https://mlsanity.com/paper/2403.12910/) (2024)
- [FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning](https://mlsanity.com/paper/2401.08553/) (2024)
- [GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration](https://mlsanity.com/paper/2311.12015/) (2023)
- [Vision-Language Models Can Self-Improve Reasoning via Reflection](https://mlsanity.com/paper/2411.00855/) (2024)
- [The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models](https://mlsanity.com/paper/2401.05618/) (2024)
- [ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning](https://mlsanity.com/paper/2410.02052/) (2024)
- [Multi-Stage Cable Routing through Hierarchical Imitation Learning](https://mlsanity.com/paper/2307.08927/) (2023)
- [Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image](https://mlsanity.com/paper/2006.05398/) (2020)

## Built on by (most cited first, 30 of 42)

- [A Survey on Vision-Language-Action Models: An Action Tokenization Perspective](https://mlsanity.com/paper/2507.01925/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents](https://mlsanity.com/paper/2505.23450/) (2025)
- [Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models](https://mlsanity.com/paper/2601.19834/) (2026)
- [AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models](https://mlsanity.com/paper/2511.14148/) (2025)
- [Current Agents Fail to Leverage World Model as Tool for Foresight](https://mlsanity.com/paper/2601.03905/) (2026)
- [CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2601.02295/) (2026)
- [LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies](https://mlsanity.com/paper/2602.21531/) (2026)
- [Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search](https://mlsanity.com/paper/2508.12211/) (2025)
- [Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs](https://mlsanity.com/paper/2602.21198/) (2026)
- [Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots](https://mlsanity.com/paper/2511.00917/) (2025)
- [Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulation](https://mlsanity.com/paper/2603.04466/) (2026)
- [REMAC: Self-Reflective and Self-Evolving Multi-Agent Collaboration for Long-Horizon Robot Manipulation](https://mlsanity.com/paper/2503.22122/) (2025)
- [PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly](https://mlsanity.com/paper/2506.08708/) (2025)
- [ETA: A New Agentic Paradigm for Embodied Tasks](https://mlsanity.com/paper/2608.03924/) (2026)
- [Simulation to Rules: A Dual-VLM Framework for Formal Visual Planning](https://mlsanity.com/paper/2510.03182/) (2025)
- [Seeing Farther and Smarter: Value-Guided Multi-Path Reflection for VLM Policy Optimization](https://mlsanity.com/paper/2602.19372/) (2026)
- [Recursive Belief Vision Language Action Models](https://mlsanity.com/paper/2602.20659/) (2026)
- [TRACE: A Self-Improving Framework for Robot Behavior Forecasting with Vision-Language Models](https://mlsanity.com/paper/2503.00761/) (2025)
- [From Instruction to Event: Sound-Triggered Mobile Manipulation](https://mlsanity.com/paper/2601.21667/) (2026)
- [PhysMem: Scaling Test-Time Memory for Embodied Physical Reasoning](https://mlsanity.com/paper/2602.20323/) (2026)
- [From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation](https://mlsanity.com/paper/2605.12167/) (2026)
- [Risk-Sensitive Agent Compositions](https://mlsanity.com/paper/2506.04632/) (2025)
- [StepWrite: Adaptive Planning for Speech-Driven Text Generation](https://mlsanity.com/paper/2508.04011/) (2025)
- [Long-Horizon Visual Imitation Learning via Plan and Code Reflection](https://mlsanity.com/paper/2509.05368/) (2025)
- [RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning](https://mlsanity.com/paper/2510.14828/) (2025)
- [Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Model](https://mlsanity.com/paper/2512.08188/) (2025)
- [ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration](https://mlsanity.com/paper/2604.04664/) (2026)
- [Task-Aware Scanning Parameter Configuration for Robotic Inspection Using Vision Language Embeddings and Hyperdimensional Computing](https://mlsanity.com/paper/2605.03909/) (2026)

## Abstract (arXiv)

Solving complex long-horizon robotic manipulation problems requires sophisticated high-level planning capabilities, the ability to reason about the physical world, and reactively choose appropriate motor skills. Vision-language models (VLMs) pretrained on Internet data could in principle offer a framework for tackling such problems. However, in their current form, VLMs lack both the nuanced understanding of intricate physics required for robotic manipulation and the ability to reason over long horizons to address error compounding issues. In this paper, we introduce a novel test-time computation framework that enhances VLMs' physical reasoning capabilities for multi-stage manipulation tasks. At its core, our approach iteratively improves a pretrained VLM with a "reflection" mechanism - it uses a generative model to imagine future world states, leverages these predictions to guide action selection, and critically reflects on potential suboptimalities to refine its reasoning. Experimental results demonstrate that our method significantly outperforms several state-of-the-art commercial VLMs as well as other post-training approaches such as Monte Carlo Tree Search (MCTS). Videos are available at https://reflect-vlm.github.io.