mlsanityData to 7 Oct 2026

Stochastic Adversarial Video Prediction

arXiv 1804.01523 ↗First posted 4 Apr 2018cs.CV cs.AI cs.LGcited by 173

Not on the map. Why?

Abstract (arXiv)

Being able to predict what may happen in the future requires an in-depth understanding of the physical and causal rules that govern the world. A model that is able to do so has a number of appealing applications, from robotic planning to representation learning. However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction. Recently, this has been addressed by two distinct approaches: (a) latent variational variable models that explicitly model underlying stochasticity and (b) adversarially-trained models that aim to produce naturalistic images. However, a standard latent variable model can struggle to produce realistic results, and a standard adversarially-trained model underutilizes latent variables and fails to produce diverse predictions. We show that these distinct methods are in fact complementary. Combining the two produces predictions that look more realistic to human raters and better cover the range of possible futures. Our method outperforms prior and concurrent work in these aspects.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 51; 2022: 44; 2023: 31; 2024: 23; 2025: 12; 2026: 12.

173 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  5. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  6. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  7. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  8. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  9. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  10. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  11. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  12. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
38 more
  1. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  3. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  4. 2023Structured World Models from Human Videoscited by 144
  5. 2021Taming Visually Guided Sound Generationcited by 136
  6. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  7. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  8. 2025Video World Models with Long-term Spatial Memorycited by 124
  9. 2022Generating Long Videos of Dynamic Scenescited by 113
  10. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  11. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  12. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  13. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  14. 2023Generative Image Dynamicscited by 93
  15. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  16. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
  17. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  18. 2023Long-Term Photometric Consistent Novel View Synthesis with Diffusion Modelscited by 76
  19. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  20. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  21. 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
  22. 2022Synthetic Data -- what, why and how?cited by 66
  23. 2021Playable Video Generationcited by 63
  24. 2021Video Prediction Recalling Long-term Motion Context via Memory Alignment Learningcited by 62
  25. 2021Predicting Video with VQVAEcited by 61
  26. 2024Probabilistic Forecasting with Stochastic Interpolants and F"ollmer Processescited by 61
  27. 2023DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcastingcited by 57
  28. 2021MotionRNN: A Flexible Model for Video Prediction with Spacetime-Varying Motionscited by 54
  29. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 54
  30. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 54
  31. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  32. 2024DINO-Foresight: Looking into the Future with DINOcited by 44
  33. 2022Temporally Consistent Transformers for Video Generationcited by 43
  34. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  35. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
  36. 2021Clockwork Variational Autoencoderscited by 40
  37. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  38. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37

50 most cited of 173 All 173 (JSON)