mlsanityData to 7 Oct 2026

LTX-Video: Realtime Video Latent Diffusion

arXiv 2501.00103 ↗First posted 30 Dec 2024cs.CVcited by 549

On the map: Diffusion models → Video generation

Abstract (arXiv)

We introduce LTX-Video, a transformer-based latent diffusion model that adopts a holistic approach to video generation by seamlessly integrating the responsibilities of the Video-VAE and the denoising transformer. Unlike existing methods, which treat these components as independent, LTX-Video aims to optimize their interaction for improved efficiency and quality. At its core is a carefully designed Video-VAE that achieves a high compression ratio of 1:192, with spatiotemporal downscaling of 32 x 32 x 8 pixels per token, enabled by relocating the patchifying operation from the transformer's input to the VAE's input. Operating in this highly compressed latent space enables the transformer to efficiently perform full spatiotemporal self-attention, which is essential for generating high-resolution videos with temporal consistency. However, the high compression inherently limits the representation of fine details. To address this, our VAE decoder is tasked with both latent-to-pixel conversion and the final denoising step, producing the clean result directly in pixel space. This approach preserves the ability to generate fine details without incurring the runtime cost of a separate upsampling module. Our model supports diverse use cases, including text-to-video and image-to-video generation, with both capabilities trained simultaneously. It achieves faster-than-real-time generation, producing 5 seconds of 24 fps video at 768x512 resolution in just 2 seconds on an Nvidia H100 GPU, outperforming all existing models of similar scale. The source code and pre-trained models are publicly available, setting a new benchmark for accessible and scalable video generation.

Key facts. Cited by 549 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 15 of its 37 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

15 of 37 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Pyramidal Flow Matching for Efficient Video Generative Modeling8 Oct
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  4. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 10

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  6. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  7. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  8. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  9. 2024FiT: Flexible Vision Transformer for Diffusion Modelcited by 84
  10. 2024Pixel-Space Post-Training of Latent Diffusion Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 211; 2026: 338.

549 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  3. 2025VACE: All-in-One Video Creation and Editingcited by 467
  4. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
  5. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  6. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  7. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  8. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  9. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  10. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  11. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  12. 2026Advancing Open-source World Modelscited by 144
38 more
  1. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  2. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  3. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  4. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  5. 2025Video World Models with Long-term Spatial Memorycited by 124
  6. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  7. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
  8. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  9. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  10. 2025Mixture of Contexts for Long Video Generationcited by 90
  11. 2025Latent Diffusion Model without Variational Autoencodercited by 83
  12. 2025Seaweed-7B: Cost-Effective Training of Video Generation Foundation Modelcited by 81
  13. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 78
  14. 2025Improving the Diffusability of Autoencoderscited by 74
  15. 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
  16. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Modelscited by 67
  17. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 64
  18. 2025Long-Context State-Space Video World Modelscited by 59
  19. 2026Helios: Real Real-Time Long Video Generation Modelcited by 56
  20. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
  21. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  22. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  23. 2025Captain Cinema: Towards Short Movie Generationcited by 48
  24. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  25. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 47
  26. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  27. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 45
  28. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  29. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  30. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 44
  31. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  32. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 43
  33. 2025MiniMax-Remover: Taming Bad Noise Helps Video Object Removalcited by 42
  34. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 40
  35. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  36. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 39
  37. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  38. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 37

50 most cited of 549 All 549 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Latte: Latent Diffusion Transformer for Video Generation
  2. 2025VGDFR: Diffusion-based Video Generation with Dynamic Latent Frame Rate
  3. 2025Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability
  4. 2025DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
  6. 2023Photorealistic Video Generation with Diffusion Models
  7. 2024OD-VAE: An Omni-dimensional Video Compressor for Improving Latent Video Diffusion Model
  8. 2024Improved Video VAE for Latent Video Diffusion Model