mlsanityData to 14 Oct 2026

VBench: Comprehensive Benchmark Suite for Video Generative Models

arXiv 2311.17982 ↗First posted 29 Nov 2023cs.CVcited by 1,822

Not on the map. Why?

Cited by 708 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Video generation has witnessed significant advancements, yet evaluating these models remains a challenge. A comprehensive evaluation benchmark for video generation is indispensable for two reasons: 1) Existing metrics do not fully align with human perceptions; 2) An ideal evaluation system should provide insights to inform future developments of video generation. To this end, we present VBench, a comprehensive benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods. VBench has three appealing properties: 1) Comprehensive Dimensions: VBench comprises 16 dimensions in video generation (e.g., subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationship, etc). The evaluation metrics with fine-grained levels reveal individual models' strengths and weaknesses. 2) Human Alignment: We also provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception, for each evaluation dimension respectively. 3) Valuable Insights: We look into current models' ability across various evaluation dimensions, and various content types. We also investigate the gaps between video and image generation models. We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.

Key facts. Cited by 1,822 papers in the map (data through 14 Oct 2026). Builds on 118 of its 145 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

118 of 145 references.

References · 118

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2017Neural Discrete Representation Learningcited by 4,712
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  7. 2016Improved Techniques for Training GANscited by 2,970
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  10. 2022Video Diffusion Modelscited by 2,305
  11. 2014Conditional Generative Adversarial Netscited by 2,162
  12. 2017The Kinetics Human Action Video Datasetcited by 1,909
  13. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  14. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  15. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  16. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  17. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  18. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  19. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  20. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  21. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  22. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  23. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  24. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  25. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  26. 2023ModelScope Text-to-Video Technical Reportcited by 658
  27. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  28. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  29. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  30. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  31. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  32. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  33. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  34. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  35. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  36. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  37. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  38. 2022MAGVIT: Masked Generative Video Transformercited by 346

50 most cited of 118 All 118 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 215; 2025: 659; 2026: 944.

1,822 papers.

Sort:

By area:

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  2. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  4. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  5. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  6. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  7. 2025VACE: All-in-One Video Creation and Editingcited by 479
  8. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  9. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  10. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  11. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  12. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
38 more
  1. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  2. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  3. 2025Improving Video Generation with Human Feedbackcited by 284
  4. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  5. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  6. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  7. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  8. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  9. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  10. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
  11. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 217
  12. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  13. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  14. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  15. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 199
  16. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  17. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  18. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
  19. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 188
  20. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  21. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  22. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  23. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  24. 2025History-Guided Video Diffusioncited by 171
  25. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  26. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  27. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
  28. 2025Do generative video models understand physical principles?cited by 161
  29. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  30. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
  31. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 156
  32. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  33. 2026Advancing Open-source World Modelscited by 154
  34. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 151
  35. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
  36. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  37. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  38. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143

50 most cited of 1,822 All 1,822 (JSON)