mlsanityData to 7 Oct 2026

Towards Accurate Generative Models of Video: A New Metric & Challenges

arXiv 1812.01717 ↗First posted 3 Dec 2018cs.CV cs.AI cs.LGcited by 1,204

Not on the map. Why?

Cited by 374 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in deep generative models have lead to remarkable progress in synthesizing high quality images. Following their successful application in image processing and representation learning, an important next step is to consider videos. Learning generative models of video is a much harder task, requiring a model to capture the temporal dynamics of a scene, in addition to the visual presentation of objects. While recent attempts at formulating generative models of video have had some success, current progress is hampered by (1) the lack of qualitative metrics that consider visual quality, temporal coherence, and diversity of samples, and (2) the wide gap between purely synthetic video data sets and challenging real-world data sets in terms of complexity. To this extent we propose Fr'echet Video Distance (FVD), a new metric for generative models of video, and StarCraft 2 Videos (SCV), a benchmark of game play from custom starcraft 2 scenarios that challenge the current capabilities of generative models of video. We contribute a large-scale human study, which confirms that FVD correlates well with qualitative human judgment of generated videos, and provide initial benchmark results on SCV.

Key facts. Cited by 1,204 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 52; 2023: 106; 2024: 269; 2025: 377; 2026: 356.

1,204 papers.

Sort:

By area:

  1. 2022Video Diffusion Modelscited by 2,280
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  6. 2023ModelScope Text-to-Video Technical Reportcited by 651
  7. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  8. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  9. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  10. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  12. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
38 more
  1. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  2. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  3. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  4. 2022MAGVIT: Masked Generative Video Transformercited by 341
  5. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  6. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  7. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  8. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  9. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  10. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  11. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  12. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  13. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  14. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  15. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  16. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  17. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  18. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  19. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  20. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  21. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  22. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  23. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  24. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  25. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  26. 2023A Survey on Video Diffusion Modelscited by 203
  27. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  28. 2025Unified Video Action Modelcited by 195
  29. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  30. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
  31. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  32. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  33. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  34. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  35. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  36. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  37. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 177
  38. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173

50 most cited of 1,204 All 1,204 (JSON)