mlsanityData to 14 Oct 2026

Transformation-based Adversarial Video Prediction on Large-Scale Data

arXiv 2003.04035 ↗First posted 9 Mar 2020cs.CV cs.LGcited by 66

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent breakthroughs in adversarial generative modeling have led to models capable of producing video samples of high quality, even on large and complex datasets of real-world video. In this work, we focus on the task of video prediction, where given a sequence of frames extracted from a video, the goal is to generate a plausible future sequence. We first improve the state of the art by performing a systematic empirical study of discriminator decompositions and proposing an architecture that yields faster convergence and higher performance than previous approaches. We then analyze recurrent units in the generator, and propose a novel recurrent unit which transforms its past hidden state according to predicted motion-like features, and refines it to handle dis-occlusions, scene changes and other complex behavior. We show that this recurrent unit consistently outperforms previous designs. Our final model leads to a leap in the state-of-the-art performance, obtaining a test set Frechet Video Distance of 25.7, down from 69.2, on the large-scale Kinetics-600 dataset.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 25; 2023: 14; 2024: 11; 2025: 6; 2026: 3.

66 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  4. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  5. 2024Genie: Generative Interactive Environmentscited by 747
  6. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  7. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  8. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  9. 2022MAGVIT: Masked Generative Video Transformercited by 346
  10. 2021Skillful Precipitation Nowcasting using Deep Generative Models of Radarcited by 311
  11. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  12. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
38 more
  1. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  2. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  3. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  4. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  5. 2022SimVP: Simpler yet Better Video Predictioncited by 184
  6. 2022Diffusion Models for Video Prediction and Infillingcited by 135
  7. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
  8. 2022Generating Long Videos of Dynamic Scenescited by 113
  9. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  10. 2024Rolling Diffusion Modelscited by 104
  11. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  12. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
  13. 2021CCVS: Context-aware Controllable Video Synthesiscited by 69
  14. 2021Playable Video Generationcited by 64
  15. 2021Predicting Video with VQVAEcited by 61
  16. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 55
  17. 2022Temporally Consistent Transformers for Video Generationcited by 44
  18. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 42
  19. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  20. 2022Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videoscited by 39
  21. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37
  22. 2022HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generatorcited by 22
  23. 2025Beyond the Frame: Generating 360 Panoramic Videos from Perspective Videoscited by 16
  24. 2023Video Diffusion Models with Local-Global Context Guidancecited by 15
  25. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  26. 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generationcited by 14
  27. 2023Sign Language Production with Latent Motion Transformercited by 13
  28. 2022Towards Smooth Video Compositioncited by 12
  29. 2023MOSO: Decomposing MOtion, Scene and Object for Video Predictioncited by 12
  30. 2023Deterministic Guidance Diffusion Model for Probabilistic Weather Forecastingcited by 9
  31. 2026Improving Flow Matching by Aligning Flow Divergencecited by 9
  32. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  33. 2022INR-V: A Continuous Representation Space for Video-based Generative Taskscited by 7
  34. 2023Learn the Force We Can: Enabling Sparse Motion Control in Multi-Object Video Generationcited by 7
  35. 2023Semantic Complete Scene Forecasting from a 4D Dynamic Point Cloud Sequencecited by 6
  36. 2024Extreme Video Compression with Pre-trained Diffusion Modelscited by 6
  37. 2022WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Predictioncited by 5
  38. 2025Masked Generative Nested Transformers with Decode Time Scalingcited by 4

50 most cited of 66 All 66 (JSON)