mlsanityData to 14 Oct 2026

Action Images: End-to-End Policy Learning via Multiview Video Generation

arXiv 2604.06168 ↗First posted 7 Apr 2026cs.CV cs.ROcited by 12

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World action models (WAMs) have emerged as a promising direction for robot policy learning, as they can leverage powerful video backbones to model the future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of video models and limiting transfer across viewpoints and environments. In this work, we present Action Images, a unified world action model that formulates policy learning as multiview video generation. Instead of encoding control as low-dimensional tokens, we translate 7-DoF robot actions into interpretable action images: multi-view action videos that are grounded in 2D pixels and explicitly track robot-arm motion. This pixel-grounded action representation allows the video backbone itself to act as a zero-shot policy, without a separate policy head or action module. Beyond control, the same unified model supports video-action joint generation, action-conditioned video generation, and action labeling under a shared representation. On RLBench and real-world evaluations, our model achieves the strongest zero-shot success rates and improves video-action joint generation quality over prior video-space world models, suggesting that interpretable action images are a promising route to policy learning.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 63 of its 71 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

63 of 71 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026World Action Models are Zero-shot Policies17 Feb
  2. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulation11 Oct
  3. 2025Learning Primitive Embodied World Models: Towards Scalable Robotic Learning28 Aug
  4. 2025Geometry-aware 4D Video Generation for Robot Manipulation1 Jul
  5. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
  6. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation15 May
  7. 2025TesserAct: Learning 4D Embodied World Models29 Apr
  8. 2025Aether: Geometric-Aware Unified World Modeling24 Mar
  9. 2024RoboDreamer: Learning Compositional World Models for Robot Imagination18 Apr
  10. 20243D-VLA: A 3D Vision-Language-Action Generative World Model14 Mar
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 52

  1. 2022Classifier-Free Diffusion Guidancecited by 5,847
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  12. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
38 more
  1. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  2. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  4. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  5. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  6. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  7. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  8. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  9. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  10. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  11. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  12. 2024Navigation World Modelscited by 272
  13. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  14. 2023Text-To-4D Dynamic Scene Generationcited by 205
  15. 2025Unified Video Action Modelcited by 205
  16. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  17. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  18. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
  19. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  20. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  21. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  22. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  23. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
  24. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  25. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 133
  26. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  27. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidancecited by 106
  28. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  29. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  30. 2025Video Generators are Robot Policiescited by 92
  31. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  32. 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 61
  33. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticscited by 51
  34. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44
  35. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 44
  36. 2025BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learningcited by 38
  37. 2025Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentationcited by 31
  38. 2025Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Modelscited by 17

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model
  2. 2026GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch
  3. 2026Vid2WAM: Distilling Video Diffusion Priors into World Action Models
  4. 2026DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning
  5. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  6. 2026Native Action-Prior Learning from Videos for World Action Models
  7. 2025CoVAR: Co-generation of Video and Action for Robotic Manipulation via Multi-Modal Diffusion
  8. 2026World2Act: Latent Action Post-Training from World Model Dynamics