mlsanityData to 7 Oct 2026

T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation

arXiv 2407.14505 ↗First posted 19 Jul 2024cs.CVcited by 177

Not on the map. Why?

Cited by 45 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-to-video (T2V) generative models have advanced significantly, yet their ability to compose different objects, attributes, actions, and motions into a video remains unexplored. Previous text-to-video benchmarks also neglect this important ability for evaluation. In this work, we conduct the first systematic study on compositional text-to-video generation. We propose T2V-CompBench, the first benchmark tailored for compositional text-to-video generation. T2V-CompBench encompasses diverse aspects of compositionality, including consistent attribute binding, dynamic attribute binding, spatial relationships, motion binding, action binding, object interactions, and generative numeracy. We further carefully design evaluation metrics of multimodal large language model (MLLM)-based, detection-based, and tracking-based metrics, which can better reflect the compositional text-to-video generation quality of seven proposed categories with 1400 text prompts. The effectiveness of the proposed metrics is verified by correlation with human evaluations. We also benchmark various text-to-video generative models and conduct in-depth analysis across different models and various compositional categories. We find that compositional text-to-video generation is highly challenging for current models, and we hope our attempt could shed light on future research in this direction.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Builds on 74 of its 96 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

74 of 96 references.

References · 74

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Segment Anythingcited by 8,158
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
38 more
  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  2. 2017The Kinetics Human Action Video Datasetcited by 1,894
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  8. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  11. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  12. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  13. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  14. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  15. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  16. 2023ModelScope Text-to-Video Technical Reportcited by 651
  17. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  18. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  19. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  20. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  21. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  22. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  23. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  24. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  25. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  26. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  27. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  28. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  29. 2022MAGVIT: Masked Generative Video Transformercited by 341
  30. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  31. 2018A Short Note about Kinetics-600cited by 331
  32. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  33. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  34. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  35. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  36. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  37. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  38. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 58; 2026: 99.

177 papers.

Sort:

By area:

  1. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  2. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  3. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  4. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  5. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  6. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  7. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  8. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  9. 2025LongCat-Video Technical Reportcited by 73
  10. 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 68
  11. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  12. 2025Video-Bench: Human-Aligned Video Generation Benchmarkcited by 53
38 more
  1. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  2. 2024Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Modelscited by 41
  3. 2025The Amazon Nova Family of Models: Technical Report and Model Cardcited by 41
  4. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 39
  5. 2025T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generationcited by 35
  6. 2025"PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Modelscited by 34
  7. 2025Judge Anything: MLLM as a Judge Across Any Modalitycited by 33
  8. 2024The Dawn of Video Generation: Preliminary Explorations with SORA-like Modelscited by 30
  9. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  10. 2024Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generationcited by 28
  11. 2025IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessmentcited by 27
  12. 2025Training-free Diffusion Acceleration with Bottleneck Samplingcited by 25
  13. 2025Generative Physical AI in Vision: A Surveycited by 24
  14. 2025Mask2DiT: Dual Mask-based Diffusion Transformer for Multi-Scene Long Video Generationcited by 24
  15. 2025CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughnesscited by 23
  16. 2025VideoGen-Eval: Agent-based System for Video Generation Evaluationcited by 23
  17. 2025The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generationcited by 21
  18. 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmarkcited by 19
  19. 2025MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesiscited by 18
  20. 2025T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generationcited by 18
  21. 2025Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Countingcited by 17
  22. 2025BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representationscited by 15
  23. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 15
  24. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  25. 2025Is Your Video Language Model a Reliable Judge?cited by 14
  26. 2024Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinementcited by 13
  27. 2025Position: Interactive Generative Video as Next-Generation Game Enginecited by 13
  28. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 12
  29. 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligencecited by 12
  30. 2025BWCache: Accelerating Video Diffusion Transformers through Block-Wise Cachingcited by 11
  31. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 11
  32. 2026LPM 1.0: Video-based Character Performance Modelcited by 11
  33. 2025Can You Count to Nine? A Human Evaluation Benchmark for Counting Limits in Modern Text-to-Video Modelscited by 10
  34. 2025UI2V-Bench: An Understanding-based Image-to-video Generation Benchmarkcited by 10
  35. 20254DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Modelscited by 10
  36. 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 10
  37. 2026RISE-Video: Can Video Generators Decode Implicit World Rules?cited by 10
  38. 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environmentscited by 10

50 most cited of 177 All 177 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards A Better Metric for Text-to-Video Generation
  2. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
  3. 2025VideoVerse: Does Your T2V Generator Have World Model Capability to Synthesize Videos?
  4. 2025SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models
  5. 2026AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation
  6. 2024GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration
  7. 2023DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation
  8. 2025Video Text Preservation with Synthetic Text-Rich Videos