# Owl-1: Omni World Model for Consistent Long Video Generation

arXiv: https://arxiv.org/abs/2412.09600 · HTML: https://mlsanity.com/paper/2412.09600/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 39 of its 42 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 39)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers](https://mlsanity.com/paper/2402.19479/) (2024)
- [I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models](https://mlsanity.com/paper/2311.04145/) (2023)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text](https://mlsanity.com/paper/2403.14773/) (2024)
- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)

## Built on by (most cited first, 18 of 18)

- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusion](https://mlsanity.com/paper/2507.00162/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [Vidarc: Embodied Video Diffusion Model for Closed-loop Control](https://mlsanity.com/paper/2512.17661/) (2025)
- [SpectralAR: Spectral Autoregressive Visual Generation](https://mlsanity.com/paper/2506.10962/) (2025)
- [BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation](https://mlsanity.com/paper/2511.22973/) (2025)
- [AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generation](https://mlsanity.com/paper/2506.03126/) (2025)
- [GenWorld: Towards Detecting AI-generated Real-world Simulation Videos](https://mlsanity.com/paper/2506.10975/) (2025)
- [GenHSI: Controllable Generation of Human-Scene Interaction Videos](https://mlsanity.com/paper/2506.19840/) (2025)
- [A Mechanistic View on Video Generation as World Models: State and Dynamics](https://mlsanity.com/paper/2601.17067/) (2026)
- [Building Social World Models with Large Language Models](https://mlsanity.com/paper/2606.11482/) (2026)
- [Terra: Explorable Native 3D World Model with Point Latents](https://mlsanity.com/paper/2510.14977/) (2025)
- [VTok: A Unified Video Tokenizer with Decoupled Spatial-Temporal Latents](https://mlsanity.com/paper/2602.04202/) (2026)
- [Narrative Weaver: Towards Controllable Long-Range Visual Consistency with Multi-Modal Conditioning](https://mlsanity.com/paper/2603.06688/) (2026)
- [World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Tasks](https://mlsanity.com/paper/2605.19957/) (2026)
- [TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization](https://mlsanity.com/paper/2606.17590/) (2026)
- [ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single View](https://mlsanity.com/paper/2509.23008/) (2025)
- [Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation](https://mlsanity.com/paper/2604.17428/) (2026)

## Abstract (arXiv)

Video generation models (VGMs) have received extensive attention recently and serve as promising candidates for general-purpose large vision models. While they can only generate short videos each time, existing methods achieve long video generation by iteratively calling the VGMs, using the last-frame output as the condition for the next-round generation. However, the last frame only contains short-term fine-grained information about the scene, resulting in inconsistency in the long horizon. To address this, we propose an Omni World modeL (Owl-1) to produce long-term coherent and comprehensive conditions for consistent long video generation. As videos are observations of the underlying evolving world, we propose to model the long-term developments in a latent space and use VGMs to film them into videos. Specifically, we represent the world with a latent state variable which can be decoded into explicit video observations. These observations serve as a basis for anticipating temporal dynamics which in turn update the state variable. The interaction between evolving dynamics and persistent state enhances the diversity and consistency of the long videos. Extensive experiments show that Owl-1 achieves comparable performance with SOTA methods on VBench-I2V and VBench-Long, validating its ability to generate high-quality video observations. Code: https://github.com/huang-yh/Owl.