Rethinking Video Generation Model for the Embodied World
Not on the map. Why?
Abstract (arXiv)
Video generation models have significantly advanced embodied intelligence, unlocking new possibilities for generating diverse robot data that capture perception, reasoning, and action in the physical world. However, synthesizing high-quality videos that accurately reflect real-world robotic interactions remains challenging, and the lack of a standardized benchmark limits fair comparisons and progress. To address this gap, we introduce a comprehensive robotics benchmark, RBench, designed to evaluate robot-oriented video generation across five task domains and four distinct embodiments. It assesses both task-level correctness and visual fidelity through reproducible sub-metrics, including structural consistency, physical plausibility, and action completeness. Evaluation of 25 representative models highlights significant deficiencies in generating physically realistic robot behaviors. Furthermore, the benchmark achieves a Spearman correlation coefficient of 0.96 with human evaluations, validating its effectiveness. While RBench provides the necessary lens to identify these deficiencies, achieving physical realism requires moving beyond evaluation to address the critical shortage of high-quality training data. Driven by these insights, we introduce a refined four-stage data pipeline, resulting in RoVid-X, the largest open-source robotic dataset for video generation with 4 million annotated video clips, covering thousands of tasks and enriched with comprehensive physical property annotations. Collectively, this synergistic ecosystem of evaluation and data establishes a robust foundation for rigorous assessment and scalable training of video models, accelerating the evolution of embodied AI toward general intelligence.
Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 93 of its 105 references that are in the map. First posted on arXiv 21 Jan 2026.
Builds on
93 of 105 references.
References · 93
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
38 more
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
- 2024Genie: Generative Interactive Environmentscited by 747
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
- 2025Seed1.5-VL Technical Reportcited by 373
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
- 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
- 2024Latent Action Pretraining from Videoscited by 321
- 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
- 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
- 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 240
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
- 2025Video models are zero-shot learners and reasonerscited by 220
- 2022Interactive Language: Talking to Robots in Real Timecited by 218
- 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
- 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
- 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 207
- 2025Unified Video Action Modelcited by 205
- 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 204
- 2025World Simulation with Video Foundation Models for Physical AIcited by 200
- 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 195
- 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
50 most cited of 93 All 93 (JSON)
Built on by
Citing papers per year: 2026: 33.
33 papers.
Sort:
By area:
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
- 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
- 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 23
- 2026Action-to-Action Flow Matchingcited by 16
- 2026HumanNet: Scaling Human-centric Video Learning to One Million Hourscited by 11
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
- 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulationcited by 10
- 2026RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulationcited by 6
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 6
- 2026Data Pyramid for Embodied Manipulation: A Surveycited by 6
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
21 more
- 2026Veo-Act: Enhancing VLA Policies with Frontier Video Modelscited by 5
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generationcited by 4
- 2026StableVLA: Towards Robust Vision-Language-Action Models without Extra Datacited by 4
- 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Modelscited by 4
- 2026Reward as An Agent for Embodied World Modelscited by 3
- 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 3
- 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Taskscited by 2
- 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imaginationcited by 2
- 2026KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Groundingcited by 2
- 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysiscited by 1
- 2026TriWorldBench: A Tri-View Consistency Perspective on Embodied World Modelscited by 1
- 2026Long-WAM: Scaling the Context of World-Action Modelscited by 1
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?cited by 0
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoningcited by 0
- 2026VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinementcited by 0
- 2026PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AIcited by 0
- 2026DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Modelscited by 0
- 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generationcited by 0
- 2026PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Modelscited by 0
- 2026EVEWorld: Physical Evolution Supervision for Embodied World Modelscited by 0
- 2026WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executionscited by 0
- 2026WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions13 Oct 2026
- 2026Long-WAM: Scaling the Context of World-Action Models7 Oct 2026
- 2026EVEWorld: Physical Evolution Supervision for Embodied World Models2 Oct 2026
- 2026PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models1 Oct 2026
- 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation1 Oct 2026
- 2026DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Models25 Sep 2026
- 2026TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models22 Sep 2026
- 2026PhysAI-Bench: A Benchmark for LLM-Based Agentic Decision-Making in Autonomous UAV-Centric Physical AI20 Sep 2026
- 2026VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement2 Sep 2026
- 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models13 Aug 2026
- 2026Data Pyramid for Embodied Manipulation: A Survey27 Jul 2026
- 2026KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding22 Jul 2026
21 more
- 2026RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination15 Jul 2026
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoning14 Jul 2026
- 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation26 Jun 2026
- 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis22 Jun 2026
- 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining18 Jun 2026
- 2026Reward as An Agent for Embodied World Models18 Jun 2026
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position13 Jun 2026
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?9 Jun 2026
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?3 Jun 2026
- 2026Cosmos 3: Omnimodal World Models for Physical AI1 Jun 2026
- 2026RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation1 Jun 2026
- 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Tasks19 May 2026
- 2026StableVLA: Towards Robust Vision-Language-Action Models without Extra Data18 May 2026
- 2026HumanNet: Scaling Human-centric Video Learning to One Million Hours7 May 2026
- 2026World Model for Robot Learning: A Comprehensive Survey30 Apr 2026
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation21 Apr 2026
- 2026Veo-Act: Enhancing VLA Policies with Frontier Video Models6 Apr 2026
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms30 Mar 2026
- 2026Helios: Real Real-Time Long Video Generation Model4 Mar 2026
- 2026Action-to-Action Flow Matching7 Feb 2026
- 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation30 Jan 2026
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Positioncited by 10
- 2026RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulationcited by 6
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 6
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
- 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Modelscited by 4
- 2026Reward as An Agent for Embodied World Modelscited by 3
- 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Taskscited by 2
- 2026KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Groundingcited by 2
- 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysiscited by 1
- 2026TriWorldBench: A Tri-View Consistency Perspective on Embodied World Modelscited by 1
12 most cited of 18 All 18 (JSON)
- 2026WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions13 Oct 2026
- 2026Long-WAM: Scaling the Context of World-Action Models7 Oct 2026
- 2026EVEWorld: Physical Evolution Supervision for Embodied World Models2 Oct 2026
- 2026PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models1 Oct 2026
- 2026DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video World Models25 Sep 2026
- 2026TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models22 Sep 2026
- 2026VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement2 Sep 2026
- 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models13 Aug 2026
- 2026KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding22 Jul 2026
- 2026RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis22 Jun 2026
- 2026Reward as An Agent for Embodied World Models18 Jun 2026
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position13 Jun 2026
12 newest of 18 All 18 (JSON)
- 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 23
- 2026Action-to-Action Flow Matchingcited by 16
- 2026Action-to-Action Flow Matching7 Feb 2026
- 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation30 Jan 2026
- 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
- 2026HumanNet: Scaling Human-centric Video Learning to One Million Hourscited by 11
- 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulationcited by 10
- 2026Data Pyramid for Embodied Manipulation: A Surveycited by 6
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generationcited by 4
- 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 3
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?cited by 0
- 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generationcited by 0
- 2026Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation1 Oct 2026
- 2026Data Pyramid for Embodied Manipulation: A Survey27 Jul 2026
- 2026PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation26 Jun 2026
- 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining18 Jun 2026
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?3 Jun 2026
- 2026HumanNet: Scaling Human-centric Video Learning to One Million Hours7 May 2026
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation21 Apr 2026
- 2026Helios: Real Real-Time Long Video Generation Model4 Mar 2026