mlsanityData to 14 Oct 2026

Envision: Embodied Visual Planning via Goal-Imagery Video Diffusion

arXiv 2512.22626 ↗First posted 27 Dec 2025cs.CVcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Embodied visual planning aims to enable manipulation tasks by imagining how a scene evolves toward a desired goal and using the imagined trajectories to guide actions. Video diffusion models, through their image-to-video generation capability, provide a promising foundation for such visual imagination. However, existing approaches are largely forward predictive, generating trajectories conditioned on the initial observation without explicit goal modeling, thus often leading to spatial drift and goal misalignment. To address these challenges, we propose Envision, a diffusion-based framework that performs visual planning for embodied agents. By explicitly constraining the generation with a goal image, our method enforces physical plausibility and goal consistency throughout the generated trajectory. Specifically, Envision operates in two stages. First, a Goal Imagery Model identifies task-relevant regions, performs region-aware cross attention between the scene and the instruction, and synthesizes a coherent goal image that captures the desired outcome. Then, an Env-Goal Video Model, built upon a first-and-last-frame-conditioned video diffusion model (FL2V), interpolates between the initial observation and the goal image, producing smooth and physically plausible video trajectories that connect the start and goal states. Experiments on object manipulation and image editing benchmarks demonstrate that Envision achieves superior goal alignment, spatial consistency, and object preservation compared to baselines. The resulting visual plans can directly support downstream robotic planning and control, providing reliable guidance for embodied agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 51 of its 68 references that are in the map. First posted on arXiv 27 Dec 2025.

Builds on

51 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
38 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  4. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  5. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  9. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  12. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  13. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  14. 2023Learning Interactive Real-World Simulatorscited by 458
  15. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  16. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  17. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  18. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  19. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
  20. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
  21. 2025TesserAct: Learning 4D Embodied World Modelscited by 94
  22. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 80
  23. 2024SeedEdit: Align Image Re-Generation to Image Editingcited by 61
  24. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  25. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 41
  26. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
  27. 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
  28. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 26
  29. 2025TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulationcited by 24
  30. 2025FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generationcited by 22
  31. 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
  32. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
  33. 2025Synthetic Video Enhances Physical Fidelity in Video Synthesiscited by 15
  34. 2025Solving New Tasks by Adapting Internet Video Knowledgecited by 13
  35. 2025DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesiscited by 12
  36. 2020One-Shot Identity-Preserving Portrait Reenactmentcited by 10
  37. 2021DisUnknown: Distilling Unknown Factors for Disentanglement Learningcited by 6
  38. 2025What does really matter in image goal navigation?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty
  2. 2025Image Generation as a Visual Planner for Robotic Manipulation
  3. 2026SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution
  4. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
  5. 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
  6. 2024Generative Image as Action Models
  7. 2025Target-Aware Video Diffusion Models
  8. 2024Transformers for Image-Goal Navigation