# GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation

arXiv: https://arxiv.org/abs/2303.17870 · HTML: https://mlsanity.com/paper/2303.17870/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Pretraining is All You Need for Image-to-Image Translation](https://mlsanity.com/paper/2205.12952/) (2022)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts](https://mlsanity.com/paper/2210.15257/) (2022)
- [Paint by Word](https://mlsanity.com/paper/2103.10951/) (2021)

## Built on by (most cited first, 30 of 34)

- [FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing](https://mlsanity.com/paper/2505.03329/) (2025)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering](https://mlsanity.com/paper/2505.24417/) (2025)
- [DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design](https://mlsanity.com/paper/2310.15144/) (2023)
- [LeX-Art: Rethinking Text Generation via Scalable High-Quality Data Synthesis](https://mlsanity.com/paper/2503.21749/) (2025)
- [TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesis](https://mlsanity.com/paper/2505.17778/) (2025)
- [AMO Sampler: Enhancing Text Rendering with Overshooting](https://mlsanity.com/paper/2411.19415/) (2024)
- [Rethinking Layered Graphic Design Generation with a Top-Down Approach](https://mlsanity.com/paper/2507.05601/) (2025)
- [TextEditBench: Evaluating Reasoning-aware Text Editing Beyond Rendering](https://mlsanity.com/paper/2512.16270/) (2025)
- [Type-R: Automatically Retouching Typos for Text-to-Image Generation](https://mlsanity.com/paper/2411.18159/) (2024)
- [WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing](https://mlsanity.com/paper/2603.11593/) (2026)
- [GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Rendering](https://mlsanity.com/paper/2603.15616/) (2026)
- [Beyond Words: Advancing Long-Text Image Generation via Multimodal Autoregressive Models](https://mlsanity.com/paper/2503.20198/) (2025)
- [FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection](https://mlsanity.com/paper/2601.00535/) (2026)
- [TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards](https://mlsanity.com/paper/2605.19320/) (2026)
- [GlyphBanana: Advancing Precise Text Rendering Through Agentic Workflows](https://mlsanity.com/paper/2603.12155/) (2026)
- [DCText: Scheduled Attention Masking for Visual Text Generation via Divide-and-Conquer Strategy](https://mlsanity.com/paper/2512.01302/) (2025)
- [SteerVTE: Seamless Video Text Editing with Style and Glyph Control](https://mlsanity.com/paper/2606.23254/) (2026)
- [Beyond Flat Text: Dual Self-inherited Guidance for Visual Text Generation](https://mlsanity.com/paper/2501.05892/) (2025)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [TextPixs: Glyph-Conditioned Diffusion with Character-Aware Attention and OCR-Guided Supervision](https://mlsanity.com/paper/2507.06033/) (2025)
- [TextGuider: Training-Free Guidance for Text Rendering via Attention Alignment](https://mlsanity.com/paper/2512.09350/) (2025)
- [PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow](https://mlsanity.com/paper/2603.25738/) (2026)
- [GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors](https://mlsanity.com/paper/2609.02349/) (2026)
- [HDGlyph: A Hierarchical Disentangled Glyph-Based Framework for Long-Tail Text Rendering in Diffusion Models](https://mlsanity.com/paper/2505.06543/) (2025)
- [SkyReels-Text: Fine-Grained Font-Controllable Text Editing for Poster Design](https://mlsanity.com/paper/2511.13285/) (2025)
- [T2LSC-Bench: Benchmarking Localized Semantic Control in Text-to-Image Generation](https://mlsanity.com/paper/2609.02255/) (2026)
- [Syn3DTxt: Embedding 3D Cues for Scene Text Generation](https://mlsanity.com/paper/2505.18479/) (2025)
- [LogoDiffuser: Training-Free Multilingual Logo Generation and Stylization via Letter-Aware Attention Control](https://mlsanity.com/paper/2603.09759/) (2026)

## Abstract (arXiv)

Recent breakthroughs in the field of language-guided image generation have yielded impressive achievements, enabling the creation of high-quality and diverse images based on user instructions.Although the synthesis performance is fascinating, one significant limitation of current image generation models is their insufficient ability to generate text coherently within images, particularly for complex glyph structures like Chinese characters. To address this problem, we introduce GlyphDraw, a general learning framework aiming to endow image generation models with the capacity to generate images coherently embedded with text for any specific language.We first sophisticatedly design the image-text dataset's construction strategy, then build our model specifically on a diffusion-based image generator and carefully modify the network structure to allow the model to learn drawing language characters with the help of glyph and position information.Furthermore, we maintain the model's open-domain image synthesis capability by preventing catastrophic forgetting by using parameter-efficient fine-tuning techniques.Extensive qualitative and quantitative experiments demonstrate that our method not only produces accurate language characters as in prompts, but also seamlessly blends the generated text into the background.Please refer to our hrefhttps://1073521013.github.io/glyph-draw.github.io/project page. endabstract