mlsanityData to 14 Oct 2026

ABHINAW: A method for Automatic Evaluation of Typography within AI-Generated Images

arXiv 2409.11874 ↗First posted 18 Sep 2024cs.CV eess.IVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In the fast-evolving field of Generative AI, platforms like MidJourney, DALL-E, and Stable Diffusion have transformed Text-to-Image (T2I) Generation. However, despite their impressive ability to create high-quality images, they often struggle to generate accurate text within these images. Theoretically, if we could achieve accurate text generation in AI images in a ``zero-shot'' manner, it would not only make AI-generated images more meaningful but also democratize the graphic design industry. The first step towards this goal is to create a robust scoring matrix for evaluating text accuracy in AI-generated images. Although there are existing bench-marking methods like CLIP SCORE and T2I-CompBench++, there's still a gap in systematically evaluating text and typography in AI-generated images, especially with diffusion-based methods. In this paper, we introduce a novel evaluation matrix designed explicitly for quantifying the performance of text and typography generation within AI-generated images. We have used letter by letter matching strategy to compute the exact matching scores from the reference text to the AI generated text. Our novel approach to calculate the score takes care of multiple redundancies such as repetition of words, case sensitivity, mixing of words, irregular incorporation of letters etc. Moreover, we have developed a Novel method named as brevity adjustment to handle excess text. In addition we have also done a quantitative analysis of frequent errors arise due to frequently used words and less frequently used words. Project page is available at: https://github.com/Abhinaw3906/ABHINAW-MATRIX.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 52 of its 67 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

52 of 67 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models27 Feb
  2. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs22 Jan
  3. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  4. 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models11 Dec
  5. 2023Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis28 Nov
  6. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts16 Oct
  7. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  8. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  9. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models23 May
  10. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
  11. 2023X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models18 May
  12. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models11 Apr
8 more
  1. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis7 Apr
  2. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  3. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  4. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  5. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2016Improved Techniques for Training GANscited by 2,970
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  11. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  12. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
20 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  2. 2016Generative Adversarial Text to Image Synthesiscited by 735
  3. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  4. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  5. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  6. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  7. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  8. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  9. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  10. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  11. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  12. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
  13. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  14. 2016Learning What and Where to Drawcited by 105
  15. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 87
  16. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 77
  17. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 58
  18. 2023Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluationcited by 46
  19. 2022Human Evaluation of Text-to-Image Models on a Multi-Task Benchmarkcited by 41
  20. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Text-to-Image Synthesis: A Decade Survey
  2. 2023DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
  3. 2025UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images
  4. 2023Word-As-Image for Semantic Typography
  5. 2024Typographic Text Generation with Off-the-Shelf Diffusion Model
  6. 2024Fine-Tuning Stable Diffusion XL for Stylistic Icon Generation: A Comparison of Caption Size
  7. 2024Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
  8. 2026FontUse: A Data-Centric Approach to Style- and Use-Case-Conditioned In-Image Typography