mlsanityData to 14 Oct 2026

T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation

arXiv 2307.06350 ↗First posted 12 Jul 2023cs.CVcited by 260

Not on the map. Why?

Cited by 98 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the impressive advances in text-to-image models, they often struggle to effectively compose complex scenes with multiple objects, displaying various attributes and relationships. To address this challenge, we present T2I-CompBench++, an enhanced benchmark for compositional text-to-image generation. T2I-CompBench++ comprises 8,000 compositional text prompts categorized into four primary groups: attribute binding, object relationships, generative numeracy, and complex compositions. These are further divided into eight sub-categories, including newly introduced ones like 3D-spatial relationships and numeracy. In addition to the benchmark, we propose enhanced evaluation metrics designed to assess these diverse compositional challenges. These include a detection-based metric tailored for evaluating 3D-spatial relationships and numeracy, and an analysis leveraging Multimodal Large Language Models (MLLMs), i.e. GPT-4V, ShareGPT4v as evaluation metrics. Our experiments benchmark 11 text-to-image models, including state-of-the-art models, such as FLUX.1, SD3, DALLE-3, Pixart-α, and SD-XL on T2I-CompBench++. We also conduct comprehensive evaluations to validate the effectiveness of our metrics and explore the potential and limitations of MLLMs.

Key facts. Cited by 260 papers in the map (data through 14 Oct 2026). Builds on 53 of its 89 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

53 of 89 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2016Improved Techniques for Training GANscited by 2,970
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2021Vision Transformers for Dense Predictioncited by 1,803
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  10. 2021Grounded Language-Image Pre-trainingcited by 1,156
  11. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  12. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  13. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  14. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  15. 2016Generative Adversarial Text to Image Synthesiscited by 735
  16. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  17. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  18. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  19. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  20. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  21. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  22. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  23. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  24. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  25. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  26. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  27. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  28. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 243
  29. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  30. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
  31. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 171
  32. 2021Simple multi-dataset detectioncited by 127
  33. 2016Learning What and Where to Drawcited by 105
  34. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Modelscited by 95
  35. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 87
  36. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Modelscited by 83
  37. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  38. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesiscited by 51

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 81; 2025: 78; 2026: 85.

260 papers.

Sort:

By area:

  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  4. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  5. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  6. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  7. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  8. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 199
  9. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  10. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  11. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
  12. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 109
38 more
  1. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 104
  2. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 100
  3. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 99
  4. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 98
  5. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  6. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 87
  7. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 87
  8. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  9. 2023GenTron: Diffusion Transformers for Image and Video Generationcited by 82
  10. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
  11. 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
  12. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
  13. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 69
  14. 2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 66
  15. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  16. 2024CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matchingcited by 61
  17. 2024Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidancecited by 59
  18. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generationcited by 54
  19. 2023Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Promptscited by 52
  20. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  21. 2025CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Modelscited by 52
  22. 2024VideoTetris: Towards Compositional Text-to-Video Generationcited by 51
  23. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  24. 2024TEXGen: a Generative Diffusion Model for Mesh Texturescited by 46
  25. 2024Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Modelscited by 45
  26. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
  27. 2024MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesiscited by 44
  28. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 43
  29. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervisioncited by 40
  30. 2024MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generationcited by 38
  31. 2024Compositional Text-to-Image Generation with Dense Blob Representationscited by 38
  32. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 38
  33. 2025T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generationcited by 38
  34. 2024Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillationcited by 37
  35. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  36. 2025Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimizationcited by 37
  37. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  38. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generationcited by 33

50 most cited of 260 All 260 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty
  2. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
  3. 2026Iterative Refinement Improves Compositional Image Generation
  4. 2025UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation
  5. 2025Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation
  6. 2025Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?
  7. 2026Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization
  8. 2025Evaluating Compositional Scene Understanding in Multimodal Generative Models