mlsanityData to 7 Oct 2026

Evaluation of Text-to-Video Generation Models: A Dynamics Perspective

arXiv 2407.01094 ↗First posted 1 Jul 2024cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

Comprehensive and constructive evaluation protocols play an important role in the development of sophisticated text-to-video (T2V) generation models. Existing evaluation protocols primarily focus on temporal consistency and content continuity, yet largely ignore the dynamics of video content. Dynamics are an essential dimension for measuring the visual vividness and the honesty of video content to text prompts. In this study, we propose an effective evaluation protocol, termed DEVIL, which centers on the dynamics dimension to evaluate T2V models. For this purpose, we establish a new benchmark comprising text prompts that fully reflect multiple dynamics grades, and define a set of dynamics scores corresponding to various temporal granularities to comprehensively evaluate the dynamics of each generated video. Based on the new benchmark and the dynamics scores, we assess T2V models with the design of three metrics: dynamics range, dynamics controllability, and dynamics-based quality. Experiments show that DEVIL achieves a Pearson correlation exceeding 90% with human ratings, demonstrating its potential to advance T2V generation models. Code is available at https://github.com/MingXiangL/DEVIL.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

30 of 46 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2022Video Diffusion Modelscited by 2,280
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  10. 2017Localizing Moments in Video with Natural Languagecited by 660
  11. 2023ModelScope Text-to-Video Technical Reportcited by 651
  12. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
18 more
  1. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  2. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  3. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  4. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  5. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  6. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  7. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  8. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  9. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  10. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  11. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  12. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  13. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124
  14. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  15. 2018Localizing Moments in Video with Temporal Languagecited by 83
  16. 2022VIDM: Video Implicit Diffusion Modelscited by 78
  17. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 64
  18. 2023StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequencescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 33; 2026: 22.

66 papers.

Sort:

By area:

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  2. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  3. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  4. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  5. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  6. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
  7. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 40
  8. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38
  9. 2025T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generationcited by 35
  10. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 34
  11. 2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
  12. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
38 more
  1. 2024Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generationcited by 28
  2. 2025CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generationcited by 21
  3. 2025NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generationcited by 21
  4. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Drivingcited by 20
  5. 2025T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generationcited by 18
  6. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  7. 2024K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferencescited by 16
  8. 2024Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-bodycited by 12
  9. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 12
  10. 2025Can You Count to Nine? A Human Evaluation Benchmark for Counting Limits in Modern Text-to-Video Modelscited by 10
  11. 2025LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretationcited by 10
  12. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  13. 2026RISE-Video: Can Video Generators Decode Implicit World Rules?cited by 10
  14. 2025T2VTextBench: A Human Evaluation Benchmark for Textual Control in Video Generation Modelscited by 9
  15. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Modelscited by 9
  16. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  17. 2025ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answeringcited by 8
  18. 2026VQQA: An Agentic Approach for Video Evaluation and Quality Improvementcited by 8
  19. 2025AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmarkcited by 7
  20. 2025T2VEval: Benchmark Dataset and Objective Evaluation Method for T2V-generated Videoscited by 6
  21. 2025LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generationcited by 6
  22. 2026EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generationcited by 6
  23. 2025Dynamic-I2V: Exploring Image-to-Video Generation Models via Multimodal LLMcited by 5
  24. 2025Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generationcited by 5
  25. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  26. 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
  27. 2025VinaBench: Benchmark for Faithful and Consistent Visual Narrativescited by 4
  28. 2026VidLeaks: Membership Inference Attacks Against Text-to-Video Modelscited by 4
  29. 2026StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generationcited by 4
  30. 2025A Survey: Spatiotemporal Consistency in Video Generationcited by 3
  31. 2026PhyEduVideo: A Benchmark for Evaluating Text-to-Video Models for Physics Educationcited by 3
  32. 2026OSCBench: Benchmarking Object State Change in Text-to-Video Generationcited by 3
  33. 2026Learning Explicit Physical Parameter Control and Benchmarking for Video Generationcited by 3
  34. 2025PhysGaia: A Physics-Aware Benchmark with Multi-Body Interactions for Dynamic Novel View Synthesiscited by 2
  35. 2025Let Your Video Listen to Your Music!cited by 2
  36. 2026Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfercited by 2
  37. 2024Kahaani: A Multimodal Co-Creative Storytelling Systemcited by 1
  38. 2024EdgeOAR: Real-time Online Action Recognition On Edge Devicescited by 1

50 most cited of 66 All 66 (JSON)