mlsanityData to 14 Oct 2026

Vidar: Embodied Video Diffusion Model for Generalist Manipulation

arXiv 2507.12898 ↗First posted 17 Jul 2025cs.LG cs.AI cs.CVcited by 62

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Scaling general-purpose manipulation to new robot embodiments remains challenging: each platform typically needs large, homogeneous demonstrations, and end-to-end pixel-to-action pipelines may degenerate under background and viewpoint shifts. Based on previous advances in video-based robot control, we present Vidar, consisting of an embodied video diffusion model as the generalizable prior and a masked inverse dynamics model (MIDM) as the adapter. We leverage a video diffusion model pre-trained at Internet scale, and further continuously pre-train it for the embodied domain using 750K multi-view trajectories collected from three real-world robot platforms. For this embodied pre-training, we introduce a unified observation space that jointly encodes robot, camera, task, and scene contexts. The MIDM module learns action-relevant pixel masks without dense labels, grounding the prior into the target embodiment's action space while suppressing distractors. With only 20 minutes of human demonstrations on an unseen robot (1% of typical data), Vidar outperforms state-of-the-art baselines and generalizes to unseen tasks, backgrounds, and camera layouts. Our results suggest a scalable recipe for "one prior, many embodiments": strong, inexpensive video priors together with minimal on-robot alignment.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 38 of its 45 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

38 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation14 Nov
  2. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation10 Oct
  3. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models16 Oct

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
23 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  2. 2018World Modelscited by 1,513
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2025s1: Simple test-time scalingcited by 1,341
  5. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  6. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  7. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  8. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  9. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
  10. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  11. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  12. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  13. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  14. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  15. 2025Unified Video Action Modelcited by 205
  16. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 195
  17. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  18. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 160
  19. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 153
  20. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  21. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 55
  22. 2024Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulationcited by 53
  23. 2025RoboEngine: Plug-and-Play Robot Data Augmentation with Semantic Robot Segmentation and Background Generationcited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 60.

62 papers.

Sort:

By area:

  1. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  2. 2025Motus: A Unified Latent Action World Modelcited by 251
  3. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 139
  4. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
  5. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
  6. 2026Motubrain: An Advanced World Action Model for Robot Controlcited by 57
  7. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  8. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  9. 2026Rethinking Video Generation Model for the Embodied Worldcited by 33
  10. 2026RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalizationcited by 30
  11. 2026ABot-M0.5: Unified Mobility-and-Manipulation World Action Modelcited by 30
  12. 2026JEPA-VLA: Video Predictive Embedding is Needed for VLA Modelscited by 24
38 more
  1. 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 23
  2. 2026PlayWorld: Learning Robot World Models from Autonomous Playcited by 18
  3. 2026MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulationcited by 17
  4. 2026S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresightcited by 17
  5. 2026WALL-WM: Carving World Action Modeling at the Event Jointscited by 17
  6. 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 16
  7. 2026FlowWAM: Optical Flow as a Unified Action Representation for World Action Modelscited by 16
  8. 2026EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewardscited by 14
  9. 2026Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generationcited by 14
  10. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 13
  11. 2026MemoryWAM: Efficient World Action Modeling with Persistent Memorycited by 13
  12. 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalizationcited by 13
  13. 2026World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesiscited by 12
  14. 2026World Action Models: A Surveycited by 12
  15. 2026GeoSem-WAM: Geometry- and Semantic-Aware World Action Modelscited by 11
  16. 2026Making Foresight Actionable: Repurposing Representation Alignment in World Action Modelscited by 11
  17. 2026HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Modelscited by 10
  18. 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 10
  19. 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulationcited by 10
  20. 2026LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulationcited by 9
  21. 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planningcited by 8
  22. 2026HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulationcited by 8
  23. 2025Robotic VLA Benefits from Joint Learning with Motion Image Diffusioncited by 7
  24. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  25. 2026WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory and In-Context Learningcited by 6
  26. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  27. 2026Veo-Act: Enhancing VLA Policies with Frontier Video Modelscited by 5
  28. 2026Temporal Backtracking Search for Test-time Generative Video Reasoningcited by 4
  29. 2026From World Models to World Action Models: A Concise Tutorial for Roboticscited by 4
  30. 2026CKT-WAM: Parameter-Efficient Context Knowledge Transfer Between World Action Modelscited by 3
  31. 2026ω-EVA: Envision, Verify, and Act with Latent Interactive World Modelscited by 3
  32. 2026Towards Predictive, Aligned, and Scalable Robot Learningcited by 3
  33. 2026BWM: A Low-Cost High-Fidelity World Simulator for Robot Learningcited by 3
  34. 2026Keep the Future, Drop the Rollout: RIFT for World Action Modelscited by 3
  35. 2026From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulationcited by 2
  36. 2026CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RLcited by 2
  37. 2026DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitationcited by 1
  38. 2026PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Explorationcited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Vidarc: Embodied Video Diffusion Model for Closed-loop Control
  2. 2026Turning Video Models into Generalist Robot Policies
  3. 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
  4. 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models
  5. 2026ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
  6. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations
  7. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training
  8. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs