mlsanityData to 7 Oct 2026

Latent Video Transformer

arXiv 2006.10704 ↗First posted 18 Jun 2020cs.CV cs.LG eess.IVcited by 106

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The video generation task can be formulated as a prediction of future video frames given some past frames. Recent generative models for videos face the problem of high computational requirements. Some models require up to 512 Tensor Processing Units for parallel training. In this work, we address this problem via modeling the dynamics in a latent space. After the transformation of frames into the latent space, our model predicts latent representation for the next frames in an autoregressive manner. We demonstrate the performance of our approach on BAIR Robot Pushing and Kinetics-600 datasets. The approach tends to reduce requirements to 8 Graphical Processing Units for training the models while maintaining comparable generation quality.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 37; 2023: 21; 2024: 20; 2025: 10; 2026: 6.

106 papers.

Sort:

By area:

  1. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  2. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  3. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  4. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  5. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  6. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  7. 2022MAGVIT: Masked Generative Video Transformercited by 341
  8. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  9. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  10. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  11. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  12. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
38 more
  1. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  2. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  3. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  4. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  5. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  6. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 146
  7. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 140
  8. 2021Taming Visually Guided Sound Generationcited by 136
  9. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  10. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  11. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  12. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  13. 2022Generating Long Videos of Dynamic Scenescited by 113
  14. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  15. 2023PreDiff: Precipitation Nowcasting with Latent Diffusion Modelscited by 96
  16. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  17. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 74
  18. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  19. 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
  20. 2022Video Transformers: A Surveycited by 63
  21. 2023Online Clustered Codebookcited by 55
  22. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
  23. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  24. 2022Temporally Consistent Transformers for Video Generationcited by 43
  25. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
  26. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  27. 2024Autoregressive Models in Vision: A Surveycited by 39
  28. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  29. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37
  30. 2022Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generationcited by 32
  31. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  32. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 30
  33. 2022EleGANt: Exquisite and Locally Editable GAN for Makeup Transfercited by 28
  34. 2025Next Block Prediction: Video Generation via Semi-Autoregressive Modelingcited by 26
  35. 2021Generative Video Transformer: Can Objects be the Words?cited by 24
  36. 2024Matten: Video Generation with Mamba-Attentioncited by 24
  37. 2022ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speechcited by 22
  38. 2022HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generatorcited by 22

50 most cited of 106 All 106 (JSON)