mlsanityData to 7 Oct 2026

Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN

arXiv 1811.09245 ↗First posted 22 Nov 2018cs.CVcited by 120

Not on the map. Why?

Cited by 40 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Training of Generative Adversarial Network (GAN) on a video dataset is a challenge because of the sheer size of the dataset and the complexity of each observation. In general, the computational cost of training GAN scales exponentially with the resolution. In this study, we present a novel memory efficient method of unsupervised learning of high-resolution video dataset whose computational cost scales only linearly with the resolution. We achieve this by designing the generator model as a stack of small sub-generators and training the model in a specific way. We train each sub-generator with its own specific discriminator. At the time of the training, we introduce between each pair of consecutive sub-generators an auxiliary subsampling layer that reduces the frame-rate by a certain ratio. This procedure can allow each sub-generator to learn the distribution of the video at different levels of resolution. We also need only a few GPUs to train a highly complex generator that far outperforms the predecessor in terms of inception scores.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 25; 2023: 36; 2024: 37; 2025: 5; 2026: 2.

120 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  7. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  8. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  9. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  10. 2022MAGVIT: Masked Generative Video Transformercited by 341
  11. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  12. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
38 more
  1. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  2. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  3. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  4. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  5. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  6. 2023A Survey on Video Diffusion Modelscited by 203
  7. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  8. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  9. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  10. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  11. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  12. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  13. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  14. 2024Training-free Camera Control for Video Generationcited by 119
  15. 2022Generating Long Videos of Dynamic Scenescited by 113
  16. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  17. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  18. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 102
  19. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  20. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 98
  21. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  22. 2022VIDM: Video Implicit Diffusion Modelscited by 78
  23. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  24. 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
  25. 2021Playable Video Generationcited by 63
  26. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 54
  27. 2023Seer: Language Instructed Video Prediction with Latent Diffusion Modelscited by 51
  28. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  29. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  30. 2021StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGANcited by 47
  31. 2024Improved Video VAE for Latent Video Diffusion Modelcited by 46
  32. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  33. 2021FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learningcited by 43
  34. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
  35. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  36. 2021CoReD: Generalizing Fake Media Detection with Continual Representation using Distillationcited by 37
  37. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  38. 2023MoStGAN-V: Video Generation with Temporal Motion Stylescited by 37

50 most cited of 120 All 120 (JSON)