Envision: Embodied Visual Planning via Goal-Imagery Video Diffusion
On the map: Diffusion models → Diffusion and flow policies for robotics
Abstract (arXiv)
Embodied visual planning aims to enable manipulation tasks by imagining how a scene evolves toward a desired goal and using the imagined trajectories to guide actions. Video diffusion models, through their image-to-video generation capability, provide a promising foundation for such visual imagination. However, existing approaches are largely forward predictive, generating trajectories conditioned on the initial observation without explicit goal modeling, thus often leading to spatial drift and goal misalignment. To address these challenges, we propose Envision, a diffusion-based framework that performs visual planning for embodied agents. By explicitly constraining the generation with a goal image, our method enforces physical plausibility and goal consistency throughout the generated trajectory. Specifically, Envision operates in two stages. First, a Goal Imagery Model identifies task-relevant regions, performs region-aware cross attention between the scene and the instruction, and synthesizes a coherent goal image that captures the desired outcome. Then, an Env-Goal Video Model, built upon a first-and-last-frame-conditioned video diffusion model (FL2V), interpolates between the initial observation and the goal image, producing smooth and physically plausible video trajectories that connect the start and goal states. Experiments on object manipulation and image editing benchmarks demonstrate that Envision achieves superior goal alignment, spatial consistency, and object preservation compared to baselines. The resulting visual plans can directly support downstream robotic planning and control, providing reliable guidance for embodied agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 51 of its 68 references that are in the map. First posted on arXiv 27 Dec 2025.
Builds on
51 of 68 references.
Closest ancestors · 1
Same direction, newest first
Other references · 50
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2023GPT-4 Technical Reportcited by 18,396
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2017Neural Discrete Representation Learningcited by 4,712
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
38 more
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
- 2023Learning Interactive Real-World Simulatorscited by 458
- 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
- 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
- 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
- 2025TesserAct: Learning 4D Embodied World Modelscited by 94
- 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 80
- 2024SeedEdit: Align Image Re-Generation to Image Editingcited by 61
- 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
- 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
- 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
- 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
- 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 26
- 2025TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulationcited by 24
- 2025FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generationcited by 22
- 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
- 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
- 2025Synthetic Video Enhances Physical Fidelity in Video Synthesiscited by 15
- 2025Solving New Tasks by Adapting Internet Video Knowledgecited by 13
- 2025DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesiscited by 12
- 2020One-Shot Identity-Preserving Portrait Reenactmentcited by 10
- 2021DisUnknown: Distilling Unknown Factors for Disentanglement Learningcited by 6
- 2025What does really matter in image goal navigation?cited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty
- 2025Image Generation as a Visual Planner for Robotic Manipulation
- 2026SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
- 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
- 2024Generative Image as Action Models
- 2025Target-Aware Video Diffusion Models
- 2024Transformers for Image-Goal Navigation