# T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation

arXiv: https://arxiv.org/abs/2407.14505 · HTML: https://mlsanity.com/paper/2407.14505/

## Key facts

- Cited by 185 papers in the map (data through 14 Oct 2026).
- Builds on 74 of its 96 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 30 of 74)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation](https://mlsanity.com/paper/2310.05737/) (2023)

## Built on by (most cited first, 30 of 185)

- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://mlsanity.com/paper/2411.19108/) (2024)
- [How Far is Video Generation from World Model: A Physical Law Perspective](https://mlsanity.com/paper/2411.02385/) (2024)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [WorldScore: A Unified Evaluation Benchmark for World Generation](https://mlsanity.com/paper/2504.00983/) (2025)
- [Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation](https://mlsanity.com/paper/2508.05635/) (2025)
- [VideoDPO: Omni-Preference Alignment for Video Diffusion Generation](https://mlsanity.com/paper/2412.14167/) (2024)
- [LongCat-Video Technical Report](https://mlsanity.com/paper/2510.22200/) (2025)
- [Allegro: Open the Black Box of Commercial-Level Video Generation Model](https://mlsanity.com/paper/2410.15458/) (2024)
- [Video-Bench: Human-Aligned Video Generation Benchmark](https://mlsanity.com/paper/2504.04907/) (2025)
- [T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design](https://mlsanity.com/paper/2410.05677/) (2024)
- [VMBench: A Benchmark for Perception-Aligned Video Motion Generation](https://mlsanity.com/paper/2503.10076/) (2025)
- [Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching](https://mlsanity.com/paper/2507.02860/) (2025)
- [Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models](https://mlsanity.com/paper/2412.09645/) (2024)
- [The Amazon Nova Family of Models: Technical Report and Model Card](https://mlsanity.com/paper/2506.12103/) (2025)
- [T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation](https://mlsanity.com/paper/2505.00337/) (2025)
- ["PhyWorldBench": A Comprehensive Evaluation of Physical Realism in Text-to-Video Models](https://mlsanity.com/paper/2507.13428/) (2025)
- [Judge Anything: MLLM as a Judge Across Any Modality](https://mlsanity.com/paper/2503.17489/) (2025)
- [Rethinking Video Generation Model for the Embodied World](https://mlsanity.com/paper/2601.15282/) (2026)
- [The Dawn of Video Generation: Preliminary Explorations with SORA-like Models](https://mlsanity.com/paper/2410.05227/) (2024)
- [Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation](https://mlsanity.com/paper/2412.16211/) (2024)
- [IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment](https://mlsanity.com/paper/2510.11647/) (2025)
- [CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness](https://mlsanity.com/paper/2502.14914/) (2025)
- [Training-free Diffusion Acceleration with Bottleneck Sampling](https://mlsanity.com/paper/2503.18940/) (2025)
- [Mask2DiT: Dual Mask-based Diffusion Transformer for Multi-Scene Long Video Generation](https://mlsanity.com/paper/2503.19881/) (2025)
- [Generative Physical AI in Vision: A Survey](https://mlsanity.com/paper/2501.10928/) (2025)
- [VideoGen-Eval: Agent-based System for Video Generation Evaluation](https://mlsanity.com/paper/2503.23452/) (2025)
- [The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation](https://mlsanity.com/paper/2504.11739/) (2025)
- [MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesis](https://mlsanity.com/paper/2503.13319/) (2025)

## Abstract (arXiv)

Text-to-video (T2V) generative models have advanced significantly, yet their ability to compose different objects, attributes, actions, and motions into a video remains unexplored. Previous text-to-video benchmarks also neglect this important ability for evaluation. In this work, we conduct the first systematic study on compositional text-to-video generation. We propose T2V-CompBench, the first benchmark tailored for compositional text-to-video generation. T2V-CompBench encompasses diverse aspects of compositionality, including consistent attribute binding, dynamic attribute binding, spatial relationships, motion binding, action binding, object interactions, and generative numeracy. We further carefully design evaluation metrics of multimodal large language model (MLLM)-based, detection-based, and tracking-based metrics, which can better reflect the compositional text-to-video generation quality of seven proposed categories with 1400 text prompts. The effectiveness of the proposed metrics is verified by correlation with human evaluations. We also benchmark various text-to-video generative models and conduct in-depth analysis across different models and various compositional categories. We find that compositional text-to-video generation is highly challenging for current models, and we hope our attempt could shed light on future research in this direction.