mlsanityData to 14 Oct 2026

Stochastic Video Generation with a Learned Prior

arXiv 1802.07687 ↗First posted 21 Feb 2018cs.CV cs.AI cs.LGcited by 245

Not on the map. Why?

Cited by 25 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generating video frames that accurately predict future world states is challenging. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce an unsupervised video generation model that learns a prior model of uncertainty in a given environment. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datasets. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

Key facts. Cited by 245 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 64; 2022: 67; 2023: 46; 2024: 28; 2025: 23; 2026: 16.

245 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  3. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  4. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  5. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  6. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  7. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  8. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  9. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  10. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  11. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  12. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
38 more
  1. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  2. 2022Diffusion Probabilistic Modeling for Video Generationcited by 238
  3. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  4. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  5. 2022SimVP: Simpler yet Better Video Predictioncited by 184
  6. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 182
  7. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  8. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  9. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  10. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  11. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
  12. 2022Diffusion Models for Video Prediction and Infillingcited by 135
  13. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  14. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  15. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  16. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  17. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
  18. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  19. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 103
  20. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  21. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
  22. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 87
  23. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 86
  24. 2023LLM-grounded Video Diffusion Modelscited by 83
  25. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  26. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  27. 2022Make-A-Story: Visual Memory Conditioned Consistent Story Generationcited by 76
  28. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  29. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  30. 2021CCVS: Context-aware Controllable Video Synthesiscited by 69
  31. 2021Video Prediction Recalling Long-term Motion Context via Memory Alignment Learningcited by 62
  32. 2023Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Treescited by 62
  33. 2021Predicting Video with VQVAEcited by 61
  34. 2025Embodied AI Agents: Modeling the Worldcited by 61
  35. 2022Implicit Motion Handling for Video Camouflaged Object Detectioncited by 60
  36. 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
  37. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 58
  38. 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 58

50 most cited of 245 All 245 (JSON)