mlsanityData to 7 Oct 2026

Generating Long Videos of Dynamic Scenes

arXiv 2206.03429 ↗First posted 7 Jun 2022cs.CV cs.AI cs.LGcited by 113

Not on the map. Why?

Cited by 39 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a video generation model that accurately reproduces object motion, changes in camera viewpoint, and new content that arises over time. Existing video generation methods often fail to produce new content as a function of time while maintaining consistencies expected in real environments, such as plausible dynamics and object persistence. A common failure case is for content to never change due to over-reliance on inductive biases to provide temporal consistency, such as a single latent code that dictates content for the entire video. On the other extreme, without long-term consistency, generated videos may morph unrealistically between different scenes. To address these limitations, we prioritize the time axis by redesigning the temporal latent representation and learning long-term consistency from data by training on longer videos. To this end, we leverage a two-phase training strategy, where we separately train using longer videos at a low resolution and shorter videos at a high resolution. To evaluate the capabilities of our model, we introduce two new benchmark datasets with explicit focus on long-term temporal dynamics.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

55 of 70 references.

References · 55

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2018SlowFast Networks for Video Recognitioncited by 1,862
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
38 more
  1. 2018World Modelscited by 1,444
  2. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  3. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  4. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  5. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  6. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  7. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  8. 2018Which Training Methods for GANs do actually Converge?cited by 672
  9. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  10. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  11. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  12. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  13. 2016Generating Videos with Scene Dynamicscited by 376
  14. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  15. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  16. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  17. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  18. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  19. 2017Stochastic Variational Video Predictioncited by 234
  20. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  21. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  22. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  23. 2018Stochastic Adversarial Video Predictioncited by 173
  24. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  25. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  26. 2017Detail-revealing Deep Video Super-resolutioncited by 147
  27. 2019Recurrent Back-Projection Network for Video Super-Resolutioncited by 147
  28. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  29. 2018Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GANcited by 120
  30. 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 118
  31. 2016Video Pixel Networkscited by 108
  32. 2020Latent Video Transformercited by 106
  33. 2017Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networkscited by 88
  34. 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
  35. 2019VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generationcited by 74
  36. 2017Recurrent Environment Simulatorscited by 68
  37. 2020Transformation-based Adversarial Video Prediction on Large-Scale Datacited by 66
  38. 2021Predicting Video with VQVAEcited by 61

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 33; 2024: 33; 2025: 33; 2026: 8.

113 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  3. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  4. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  6. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  7. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  8. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  9. 2022MAGVIT: Masked Generative Video Transformercited by 341
  10. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  11. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  12. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
38 more
  1. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  2. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  3. 2023A Survey on Video Diffusion Modelscited by 203
  4. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  5. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  6. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  7. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  8. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  9. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  10. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  11. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  12. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  13. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  14. 2023Generative Image Dynamicscited by 93
  15. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Modelscited by 93
  16. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  17. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  18. 2023LLM-grounded Video Diffusion Modelscited by 83
  19. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  20. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  21. 2024Vlogger: Make Your Dream A Vlogcited by 70
  22. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  23. 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 64
  24. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  25. 2023Text2Performer: Text-Driven Human Video Generationcited by 53
  26. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53
  27. 2024On the Content Bias in Fr'echet Video Distancecited by 51
  28. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  29. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  30. 2022Temporally Consistent Transformers for Video Generationcited by 43
  31. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  32. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
  33. 2023MEVG: Multi-event Video Generation with Text-to-Video Modelscited by 40
  34. 2024A Survey on Long Video Generation: Challenges, Methods, and Prospectscited by 40
  35. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  36. 2022Deep Latent State Space Models for Time-Series Generationcited by 37
  37. 2023MoStGAN-V: Video Generation with Temporal Motion Stylescited by 37
  38. 2024DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Frameworkcited by 31

50 most cited of 113 All 113 (JSON)