# EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models

arXiv: https://arxiv.org/abs/2505.09694 · HTML: https://mlsanity.com/paper/2505.09694/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 15 of 15)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models](https://mlsanity.com/paper/2311.04145/) (2023)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models](https://mlsanity.com/paper/2405.04233/) (2024)
- [ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation](https://mlsanity.com/paper/2402.04324/) (2024)
- [EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation](https://mlsanity.com/paper/2501.01895/) (2025)
- [EVA: An Embodied World Model for Future Video Anticipation](https://mlsanity.com/paper/2410.15461/) (2024)

## Built on by (most cited first, 30 of 50)

- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation](https://mlsanity.com/paper/2508.05635/) (2025)
- [WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models](https://mlsanity.com/paper/2602.08971/) (2026)
- [WorldEval: World Model as Real-World Robot Policies Evaluator](https://mlsanity.com/paper/2505.19017/) (2025)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [World Model for Robot Learning: A Comprehensive Survey](https://mlsanity.com/paper/2605.00080/) (2026)
- [WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation](https://mlsanity.com/paper/2605.25874/) (2026)
- [Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models](https://mlsanity.com/paper/2605.06388/) (2026)
- [DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving](https://mlsanity.com/paper/2601.01528/) (2026)
- [RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation](https://mlsanity.com/paper/2604.19092/) (2026)
- [MobileWorldBench: Towards Semantic World Modeling For Mobile Agents](https://mlsanity.com/paper/2512.14014/) (2025)
- [ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction](https://mlsanity.com/paper/2511.20937/) (2025)
- [iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework](https://mlsanity.com/paper/2605.03941/) (2026)
- [Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation](https://mlsanity.com/paper/2606.17030/) (2026)
- [Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test](https://mlsanity.com/paper/2601.04137/) (2026)
- [Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions](https://mlsanity.com/paper/2601.07823/) (2026)
- [MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation](https://mlsanity.com/paper/2512.14691/) (2025)
- [LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment](https://mlsanity.com/paper/2604.11689/) (2026)
- [WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform](https://mlsanity.com/paper/2605.17912/) (2026)
- [How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position](https://mlsanity.com/paper/2606.15032/) (2026)
- [WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models](https://mlsanity.com/paper/2606.31672/) (2026)
- [What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction](https://mlsanity.com/paper/2606.07687/) (2026)
- [OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics](https://mlsanity.com/paper/2606.04463/) (2026)
- [RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation](https://mlsanity.com/paper/2606.01600/) (2026)
- [WorldOlympiad: Can Your World Model Survive a Triathlon?](https://mlsanity.com/paper/2606.11129/) (2026)
- [Aligning Agentic World Models via Knowledgeable Experience Learning](https://mlsanity.com/paper/2601.13247/) (2026)
- [A Definition and Roadmap for World Models](https://mlsanity.com/paper/2607.06401/) (2026)
- [An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges](https://mlsanity.com/paper/2512.11362/) (2025)

## Abstract (arXiv)

Recent advances in creative AI have enabled the synthesis of high-fidelity images and videos conditioned on language instructions. Building on these developments, text-to-video diffusion models have evolved into embodied world models (EWMs) capable of generating physically plausible scenes from language commands, effectively bridging vision and action in embodied AI applications. This work addresses the critical challenge of evaluating EWMs beyond general perceptual metrics to ensure the generation of physically grounded and action-consistent behaviors. We propose the Embodied World Model Benchmark (EWMBench), a dedicated framework designed to evaluate EWMs based on three key aspects: visual scene consistency, motion correctness, and semantic alignment. Our approach leverages a meticulously curated dataset encompassing diverse scenes and motion patterns, alongside a comprehensive multi-dimensional evaluation toolkit, to assess and compare candidate models. The proposed benchmark not only identifies the limitations of existing video generation models in meeting the unique requirements of embodied tasks but also provides valuable insights to guide future advancements in the field. The dataset and evaluation tools are publicly available at https://github.com/AgibotTech/EWMBench.