# EVA: An Embodied World Model for Future Video Anticipation

arXiv: https://arxiv.org/abs/2410.15461 · HTML: https://mlsanity.com/paper/2410.15461/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 58 of its 71 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 58)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)

## Built on by (most cited first, 28 of 28)

- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [AdaWorld: Learning Adaptable World Models with Latent Actions](https://mlsanity.com/paper/2503.18938/) (2025)
- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025)
- [EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models](https://mlsanity.com/paper/2505.09694/) (2025)
- [3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model](https://mlsanity.com/paper/2506.06199/) (2025)
- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [Learning 3D Persistent Embodied World Models](https://mlsanity.com/paper/2505.05495/) (2025)
- [ORV: 4D Occupancy-centric Robot Video Generation](https://mlsanity.com/paper/2506.03079/) (2025)
- [Mask World Model: Predicting What Matters for Robust Robot Policy Learning](https://mlsanity.com/paper/2604.19683/) (2026)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction](https://mlsanity.com/paper/2511.20937/) (2025)
- [MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis](https://mlsanity.com/paper/2506.18897/) (2025)
- [Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test](https://mlsanity.com/paper/2601.04137/) (2026)
- [Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions](https://mlsanity.com/paper/2601.07823/) (2026)
- [ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance](https://mlsanity.com/paper/2504.16464/) (2025)
- [PhysicsMind: Sim and Real Mechanics Benchmarking for Physical Reasoning and Prediction in Foundational VLMs and World Models](https://mlsanity.com/paper/2601.16007/) (2026)
- [How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position](https://mlsanity.com/paper/2606.15032/) (2026)
- [TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion](https://mlsanity.com/paper/2601.18323/) (2026)
- [GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation](https://mlsanity.com/paper/2605.22882/) (2026)
- [WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time](https://mlsanity.com/paper/2607.06988/) (2026)
- [Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models](https://mlsanity.com/paper/2509.19994/) (2025)
- [Bounding Distributional Shifts in World Modeling through Novelty Detection](https://mlsanity.com/paper/2508.06096/) (2025)
- [SC2-WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2608.07548/) (2026)
- [PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control](https://mlsanity.com/paper/2509.24591/) (2025)
- [ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models](https://mlsanity.com/paper/2604.16405/) (2026)
- [HappyWorld-Bench](https://mlsanity.com/paper/2609.24308/) (2026)
- [Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence](https://mlsanity.com/paper/2609.29669/) (2026)

## Abstract (arXiv)

Video generation models have made significant progress in simulating future states, showcasing their potential as world simulators in embodied scenarios. However, existing models often lack robust understanding, limiting their ability to perform multi-step predictions or handle Out-of-Distribution (OOD) scenarios. To address this challenge, we propose the Reflection of Generation (RoG), a set of intermediate reasoning strategies designed to enhance video prediction. It leverages the complementary strengths of pre-trained vision-language and video generation models, enabling them to function as a world model in embodied scenarios. To support RoG, we introduce Embodied Video Anticipation Benchmark(EVA-Bench), a comprehensive benchmark that evaluates embodied world models across diverse tasks and scenarios, utilizing both in-domain and OOD datasets. Building on this foundation, we devise a world model, Embodied Video Anticipator (EVA), that follows a multistage training paradigm to generate high-fidelity video frames and apply an autoregressive strategy to enable adaptive generalization for longer video sequences. Extensive experiments demonstrate the efficacy of EVA in various downstream tasks like video generation and robotics, thereby paving the way for large-scale pre-trained models in real-world video prediction applications. The video demos are available at hyperlinkhttps://sites.google.com/view/icml-evahttps://sites.google.com/view/icml-eva.