mlsanityData to 7 Oct 2026

Type-R: Automatically Retouching Typos for Text-to-Image Generation

arXiv 2411.18159 ↗First posted 27 Nov 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

While recent text-to-image models can generate photorealistic images from text prompts that reflect detailed instructions, they still face significant challenges in accurately rendering words in the image. In this paper, we propose to retouch erroneous text renderings in the post-processing pipeline. Our approach, called Type-R, identifies typographical errors in the generated image, erases the erroneous text, regenerates text boxes for missing words, and finally corrects typos in the rendered words. Through extensive experiments, we show that Type-R, in combination with the latest text-to-image models such as Stable Diffusion or Flux, achieves the highest text rendering accuracy while maintaining image quality and also outperforms text-focused generation baselines in terms of balancing text accuracy and image quality.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 46 of its 54 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

46 of 54 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  8. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  9. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  10. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 278
  11. 2019Character Region Awareness for Text Detectioncited by 273
  12. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
34 more
  1. 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
  2. 2019What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysiscited by 191
  3. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
  4. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
  5. 2023GlyphControl: Glyph Conditional Control for Visual Text Generationcited by 127
  6. 2020Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spottingcited by 114
  7. 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
  8. 2022Character-Aware Models Improve Visual Text Renderingcited by 90
  9. 2019Editing Text in the Wildcited by 81
  10. 2024Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Renderingcited by 80
  11. 2022DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spottingcited by 74
  12. 2023UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Modelscited by 58
  13. 2024Harmonizing Visual Text Comprehension and Generationcited by 54
  14. 2020SwapText: Image Based Texts Transfer in Scenescited by 49
  15. 2023LayoutPrompter: Awaken the Design Ability of Large Language Modelscited by 47
  16. 2024Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual Visual Text Renderingcited by 47
  17. 2019STEFANN: Scene Text Editor using Font Adaptive Neural Networkcited by 46
  18. 2023Improving Diffusion Models for Scene Text Editing with Dual Encoderscited by 44
  19. 2024TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Controlcited by 44
  20. 2023Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Modelcited by 41
  21. 2022Exploring Stroke-Level Modifications for Scene Text Editingcited by 36
  22. 2024OpenCOLE: Towards Reproducible Automatic Graphic Design Generationcited by 36
  23. 2023GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generationcited by 34
  24. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  25. 2024Visual Text Generation in the Wildcited by 22
  26. 2023Parrot Captions Teach CLIP to Spot Textcited by 16
  27. 2024First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blendingcited by 16
  28. 2024Layout Agnostic Scene Text Image Synthesis with Diffusion Modelscited by 14
  29. 2022The Surprisingly Straightforward Scene Text Removal Method With Gated Attention and Region of Interest Generation: A Comprehensive Prominent Model Analysiscited by 13
  30. 2024Refining Text-to-Image Generation: Towards Accurate Training-Free Glyph-Enhanced Image Generationcited by 13
  31. 2024JoyType: A Robust Design for Multilingual Visual Text Creationcited by 12
  32. 2023An Empirical Study of Scaling Law for OCRcited by 7
  33. 2024Typographic Text Generation with Off-the-Shelf Diffusion Modelcited by 5
  34. 2024Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Trainingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024TypeScore: A Text Fidelity Metric for Text-to-Image Generative Models
  2. 2025LeX-Art: Rethinking Text Generation via Scalable High-Quality Data Synthesis
  3. 2025TextEditBench: Evaluating Reasoning-aware Text Editing Beyond Rendering
  4. 2025POET: Preference Optimization for Enhanced Text-to-Image Generation
  5. 2024FonTS: Text Rendering with Typography and Style Controls
  6. 2026FontUse: A Data-Centric Approach to Style- and Use-Case-Conditioned In-Image Typography
  7. 2024ABHINAW: A method for Automatic Evaluation of Typography within AI-Generated Images
  8. 2023Rich Human Feedback for Text-to-Image Generation