mlsanityData to 7 Oct 2026

T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts

arXiv 2412.04300 ↗First posted 5 Dec 2024cs.CV cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Evaluating the quality of synthesized images remains a significant challenge in the development of text-to-image (T2I) generation. Most existing studies in this area primarily focus on evaluating text-image alignment, image quality, and object composition capabilities, with comparatively fewer studies addressing the evaluation of the factuality of T2I models, particularly when the concepts involved are knowledge-intensive. To mitigate this gap, we present T2I-FactualBench in this work - the largest benchmark to date in terms of the number of concepts and prompts specifically designed to evaluate the factuality of knowledge-intensive concept generation. T2I-FactualBench consists of a three-tiered knowledge-intensive text-to-image generation framework, ranging from the basic memorization of individual knowledge concepts to the more complex composition of multiple knowledge concepts. We further introduce a multi-round visual question answering (VQA) based evaluation framework to assess the factuality of three-tiered knowledge-intensive text-to-image generation tasks. Experiments on T2I-FactualBench indicate that current state-of-the-art (SOTA) T2I models still leave significant room for improvement.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 43 of its 66 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

43 of 66 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
31 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  2. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  3. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  4. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  5. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  6. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  7. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  8. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  9. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 249
  10. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  11. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  12. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  13. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  14. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  15. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  16. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  17. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 118
  18. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
  19. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
  20. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 75
  21. 2022The Creativity of Text-to-Image Generationcited by 73
  22. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  23. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficultycited by 60
  24. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  25. 2024MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesiscited by 44
  26. 2024PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Modelscited by 36
  27. 2024LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generationcited by 30
  28. 2023"An Adapt-or-Die Type of Situation": Perception, Adoption, and Use of Text-To-Image-Generation AI by Game Industry Professionalscited by 11
  29. 2024Chaining text-to-image and large language model: A novel approach for generating personalized e-commerce bannerscited by 11
  30. 2024TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competitioncited by 10
  31. 2024KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entitiescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 4.

12 papers.

  1. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  2. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  3. 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 16
  4. 2025Science-T2I: Addressing Scientific Illusions in Image Synthesiscited by 15
  5. 2025Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Modelscited by 4
  6. 2025From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generationcited by 4
  7. 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generationcited by 3
  8. 2025TokenFocus-VQA: Enhancing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMscited by 2
  9. 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insightscited by 2
  10. 2026GenScale: A Benchmark for Relative Object Scale in Image Generation and Editingcited by 1
  11. 2026FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Modelscited by 0
  12. 2026WorldFact-Bench: Beyond Image-Internal Plausibility to Image-World Consistencycited by 0