mlsanityData to 14 Oct 2026

Rethinking Video Generation Model for the Embodied World

arXiv 2601.15282 ↗First posted 21 Jan 2026cs.CV cs.AI cs.ROcited by 33

Not on the map. Why?

Abstract (arXiv)

Video generation models have significantly advanced embodied intelligence, unlocking new possibilities for generating diverse robot data that capture perception, reasoning, and action in the physical world. However, synthesizing high-quality videos that accurately reflect real-world robotic interactions remains challenging, and the lack of a standardized benchmark limits fair comparisons and progress. To address this gap, we introduce a comprehensive robotics benchmark, RBench, designed to evaluate robot-oriented video generation across five task domains and four distinct embodiments. It assesses both task-level correctness and visual fidelity through reproducible sub-metrics, including structural consistency, physical plausibility, and action completeness. Evaluation of 25 representative models highlights significant deficiencies in generating physically realistic robot behaviors. Furthermore, the benchmark achieves a Spearman correlation coefficient of 0.96 with human evaluations, validating its effectiveness. While RBench provides the necessary lens to identify these deficiencies, achieving physical realism requires moving beyond evaluation to address the critical shortage of high-quality training data. Driven by these insights, we introduce a refined four-stage data pipeline, resulting in RoVid-X, the largest open-source robotic dataset for video generation with 4 million annotated video clips, covering thousands of tasks and enriched with comprehensive physical property annotations. Collectively, this synergistic ecosystem of evaluation and data establishes a robust foundation for rigorous assessment and scalable training of video models, accelerating the evolution of embodied AI toward general intelligence.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 93 of its 105 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

93 of 105 references.

References · 93

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2025Qwen3-VL Technical Reportcited by 2,793
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  11. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  12. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
38 more
  1. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  2. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  3. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  4. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  5. 2024Genie: Generative Interactive Environmentscited by 747
  6. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  7. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  8. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  9. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  10. 2025VACE: All-in-One Video Creation and Editingcited by 479
  11. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  12. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
  13. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  14. 2025Seed1.5-VL Technical Reportcited by 373
  15. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  16. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  17. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  18. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  19. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  20. 2024Latent Action Pretraining from Videoscited by 321
  21. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  22. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  23. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  24. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
  25. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  26. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 240
  27. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  28. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  29. 2025Video models are zero-shot learners and reasonerscited by 220
  30. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  31. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  32. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  33. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 207
  34. 2025Unified Video Action Modelcited by 205
  35. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 204
  36. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  37. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 195
  38. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 33.

33 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  3. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  4. 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 23
  5. 2026Action-to-Action Flow Matchingcited by 16
  6. 2026HumanNet: Scaling Human-centric Video Learning to One Million Hourscited by 11
  7. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
  8. 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulationcited by 10
  9. 2026RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulationcited by 6
  10. 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 6
  11. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 6
  12. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
21 more
  1. 2026Veo-Act: Enhancing VLA Policies with Frontier Video Modelscited by 5
  2. 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generationcited by 4
  3. 2026StableVLA: Towards Robust Vision-Language-Action Models without Extra Datacited by 4
  4. 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Modelscited by 4
  5. 2026Reward as An Agent for Embodied World Modelscited by 3
  6. 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 3
  7. 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Taskscited by 2
  8. 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imaginationcited by 2
  9. 2026KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Groundingcited by 2
  10. 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysiscited by 1
  11. 2026TriWorldBench: A Tri-View Consistency Perspective on Embodied World Modelscited by 1
  12. 2026Long-WAM: Scaling the Context of World-Action Modelscited by 1
  13. 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?cited by 0
  14. 2026UniVR: Thinking in Visual Space for Unified Visual Reasoningcited by 0
  15. 2026VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinementcited by 0
  16. 2026PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AIcited by 0
  17. 2026DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Modelscited by 0
  18. 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generationcited by 0
  19. 2026PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Modelscited by 0
  20. 2026EVEWorld: Physical Evolution Supervision for Embodied World Modelscited by 0
  21. 2026WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executionscited by 0