mlsanityData to 7 Oct 2026

TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

arXiv 2303.11897 ↗First posted 21 Mar 2023cs.CVcited by 416

Not on the map. Why?

Cited by 104 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite thousands of researchers, engineers, and artists actively working on improving text-to-image generation models, systems often fail to produce images that accurately align with the text inputs. We introduce TIFA (Text-to-Image Faithfulness evaluation with question Answering), an automatic evaluation metric that measures the faithfulness of a generated image to its text input via visual question answering (VQA). Specifically, given a text input, we automatically generate several question-answer pairs using a language model. We calculate image faithfulness by checking whether existing VQA models can answer these questions using the generated image. TIFA is a reference-free metric that allows for fine-grained and interpretable evaluations of generated images. TIFA also has better correlations with human judgments than existing metrics. Based on this approach, we introduce TIFA v1.0, a benchmark consisting of 4K diverse text inputs and 25K questions across 12 categories (object, counting, etc.). We present a comprehensive evaluation of existing text-to-image models using TIFA v1.0 and highlight the limitations and challenges of current models. For instance, we find that current text-to-image models, despite doing well on color and material, still struggle in counting, spatial relations, and composing multiple objects. We hope our benchmark will help carefully measure the research progress in text-to-image synthesis and provide valuable insights for further research.

Key facts. Cited by 416 papers in the map (data through 7 Oct 2026). Builds on 53 of its 64 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

53 of 64 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2019Towards VQA Models That Can Readcited by 1,696
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  11. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  13. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  14. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  15. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  16. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  17. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  18. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  19. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  20. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  21. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  22. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  23. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  24. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  25. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  26. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 177
  27. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  28. 2022PromptCap: Prompt-Guided Task-Aware Image Captioningcited by 104
  29. 2018Inferring Semantic Layout for Hierarchical Text-to-Image Synthesiscited by 101
  30. 2022Character-Aware Models Improve Visual Text Renderingcited by 90
  31. 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89
  32. 2022All You May Need for VQA are Image Captionscited by 65
  33. 2020X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformerscited by 58
  34. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  35. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  36. 2022Human Evaluation of Text-to-Image Models on a Multi-Task Benchmarkcited by 41
  37. 2019Information Maximizing Visual Question Generationcited by 40
  38. 2021Transparent Human Evaluation for Image Captioningcited by 40

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 51; 2024: 113; 2025: 132; 2026: 120.

416 papers.

Sort:

By area:

  1. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  2. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  3. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  4. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  5. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  6. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  7. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  8. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
  9. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  10. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  11. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 163
  12. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
38 more
  1. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
  2. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  3. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  4. 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 130
  5. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  6. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  7. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 118
  8. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  9. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  10. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 102
  11. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
  12. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 95
  13. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  14. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
  15. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
  16. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
  17. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  18. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 81
  19. 2023Divide & Bind Your Attention for Improved Generative Semantic Nursingcited by 80
  20. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 64
  21. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  22. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  23. 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
  24. 2024CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matchingcited by 60
  25. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficultycited by 60
  26. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  27. 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Modelscited by 54
  28. 2023Visual Programming for Text-to-Image Generation and Evaluationcited by 53
  29. 2025Video-Bench: Human-Aligned Video Generation Benchmarkcited by 53
  30. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  31. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  32. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 51
  33. 2023Revisiting the Role of Language Priors in Vision-Language Modelscited by 50
  34. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  35. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  36. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 45
  37. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 44
  38. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 44

50 most cited of 416 All 416 (JSON)