# First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending arXiv: https://arxiv.org/abs/2410.10168 ยท HTML: https://mlsanity.com/paper/2410.10168/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 31 of its 42 references that are in the map. - First posted on arXiv 14 Oct 2024. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 30 of 31) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021) - [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023) - [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021) - [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016) - [Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.04803/) (2023) - [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023) - [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019) - [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021) - [AnyText: Multilingual Visual Text Generation And Editing](https://mlsanity.com/paper/2311.03054/) (2023) - [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019) - [GlyphControl: Glyph Conditional Control for Visual Text Generation](https://mlsanity.com/paper/2305.18259/) (2023) - [LayoutTransformer: Layout Generation and Completion with Self-attention](https://mlsanity.com/paper/2006.14615/) (2020) - [PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System](https://mlsanity.com/paper/2206.03001/) (2022) - [Character-Aware Models Improve Visual Text Rendering](https://mlsanity.com/paper/2212.10562/) (2022) - [Spatial Fusion GAN for Image Synthesis](https://mlsanity.com/paper/1812.05840/) (2018) - [Improving Diffusion Models for Scene Text Editing with Dual Encoders](https://mlsanity.com/paper/2304.05568/) (2023) - [Verisimilar Image Synthesis for Accurate Detection and Recognition of Texts in Scenes](https://mlsanity.com/paper/1807.03021/) (2018) - [UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World](https://mlsanity.com/paper/2003.10608/) (2020) - [Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model](https://mlsanity.com/paper/2312.12232/) (2023) - [TPSNet: Reverse Thinking of Thin Plate Splines for Arbitrary Shape Scene Text Representation](https://mlsanity.com/paper/2110.12826/) (2021) - [Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection](https://mlsanity.com/paper/2109.03426/) (2021) - [SynthText3D: Synthesizing Scene Text Images from 3D Virtual Worlds](https://mlsanity.com/paper/1907.06007/) (2019) - [Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning](https://mlsanity.com/paper/2308.07202/) (2023) - [Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing](https://mlsanity.com/paper/2402.03082/) (2024) - [TextBlockV2: Towards Precise-Detection-Free Scene Text Spotting with Pre-trained Language Model](https://mlsanity.com/paper/2403.10047/) (2024) ## Built on by (most cited first, 16 of 16) - [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025) - [When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding](https://mlsanity.com/paper/2506.05551/) (2025) - [Arbitrary Reading Order Scene Text Spotter with Local Semantics Guidance](https://mlsanity.com/paper/2412.10159/) (2024) - [VidText: Towards Comprehensive Evaluation for Video Text Understanding](https://mlsanity.com/paper/2505.22810/) (2025) - [ControlText: Unlocking Controllable Fonts in Multilingual Text Rendering without Font Annotations](https://mlsanity.com/paper/2502.10999/) (2025) - [Type-R: Automatically Retouching Typos for Text-to-Image Generation](https://mlsanity.com/paper/2411.18159/) (2024) - [Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2503.18746/) (2025) - [Uni-DocDiff: A Unified Document Restoration Model Based on Diffusion](https://mlsanity.com/paper/2508.04055/) (2025) - [StyleTextGen: Style-Conditioned Multilingual Scene Text Generation](https://mlsanity.com/paper/2605.14708/) (2026) - [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025) - [IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation](https://mlsanity.com/paper/2603.10495/) (2026) - [UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation](https://mlsanity.com/paper/2606.24333/) (2026) - [TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation](https://mlsanity.com/paper/2608.16284/) (2026) - [Syn3DTxt: Embedding 3D Cues for Scene Text Generation](https://mlsanity.com/paper/2505.18479/) (2025) - [TADoc: Robust Time-Aware Document Image Dewarping](https://mlsanity.com/paper/2508.06988/) (2025) - [MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment](https://mlsanity.com/paper/2607.00410/) (2026) ## Abstract (arXiv) Diffusion models, known for their impressive image generation abilities, have played a pivotal role in the rise of visual text generation. Nevertheless, existing visual text generation methods often focus on generating entire images with text prompts, leading to imprecise control and limited practicality. A more promising direction is visual text blending, which focuses on seamlessly merging texts onto text-free backgrounds. However, existing visual text blending methods often struggle to generate high-fidelity and diverse images due to a shortage of backgrounds for synthesis and limited generalization capabilities. To overcome these challenges, we propose a new visual text blending paradigm including both creating backgrounds and rendering texts. Specifically, a background generator is developed to produce high-fidelity and text-free natural images. Moreover, a text renderer named GlyphOnly is designed for achieving visually plausible text-background integration. GlyphOnly, built on a Stable Diffusion framework, utilizes glyphs and backgrounds as conditions for accurate rendering and consistency control, as well as equipped with an adaptive text block exploration strategy for small-scale text rendering. We also explore several downstream applications based on our method, including scene text dataset synthesis for boosting scene text detectors, as well as text image customization and editing. Code and model will be available at urlhttps://github.com/Zhenhang-Li/GlyphOnly.