# ControlText: Unlocking Controllable Fonts in Multilingual Text Rendering without Font Annotations arXiv: https://arxiv.org/abs/2502.10999 ยท HTML: https://mlsanity.com/paper/2502.10999/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 32 of its 62 references that are in the map. - First posted on arXiv 16 Feb 2025. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 30 of 32) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024) - [Unleashing Text-to-Image Diffusion Models for Visual Perception](https://mlsanity.com/paper/2303.02153/) (2023) - [Text-to-image Diffusion Models in Generative AI: A Survey](https://mlsanity.com/paper/2303.07909/) (2023) - [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023) - [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020) - [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023) - [GlyphControl: Glyph Conditional Control for Visual Text Generation](https://mlsanity.com/paper/2305.18259/) (2023) - [Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion](https://mlsanity.com/paper/2310.03502/) (2023) - [Character-Aware Models Improve Visual Text Rendering](https://mlsanity.com/paper/2212.10562/) (2022) - [Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Rendering](https://mlsanity.com/paper/2403.09622/) (2024) - [ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer](https://mlsanity.com/paper/2410.00086/) (2024) - [Harmonizing Visual Text Comprehension and Generation](https://mlsanity.com/paper/2407.16364/) (2024) - [AnyText2: Visual Text Generation and Editing With Customizable Attributes](https://mlsanity.com/paper/2411.15245/) (2024) - [Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual Visual Text Rendering](https://mlsanity.com/paper/2406.10208/) (2024) - [RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model](https://mlsanity.com/paper/2309.00810/) (2023) - [Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach](https://mlsanity.com/paper/2011.14021/) (2020) - [DiffUTE: Universal Text Editing Diffusion Model](https://mlsanity.com/paper/2305.10825/) (2023) - [Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model](https://mlsanity.com/paper/2312.12232/) (2023) - [FonTS: Text Rendering with Typography and Style Controls](https://mlsanity.com/paper/2412.00136/) (2024) - [Visual Text Generation in the Wild](https://mlsanity.com/paper/2407.14138/) (2024) - [First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending](https://mlsanity.com/paper/2410.10168/) (2024) - [Refining Text-to-Image Generation: Towards Accurate Training-Free Glyph-Enhanced Image Generation](https://mlsanity.com/paper/2403.16422/) (2024) - [JoyType: A Robust Design for Multilingual Visual Text Creation](https://mlsanity.com/paper/2409.17524/) (2024) - [Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation](https://mlsanity.com/paper/2407.14774/) (2024) - [HFH-Font: Few-shot Chinese Font Synthesis with Higher Quality, Faster Speed, and Higher Resolution](https://mlsanity.com/paper/2410.06488/) (2024) - [A fine-grained approach to scene text script identification](https://mlsanity.com/paper/1602.07475/) (2016) ## Built on by (most cited first, 10 of 10) - [EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering](https://mlsanity.com/paper/2505.24417/) (2025) - [RepText: Rendering Visual Text via Replicating](https://mlsanity.com/paper/2504.19724/) (2025) - [InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation](https://mlsanity.com/paper/2603.05898/) (2026) - [GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Rendering](https://mlsanity.com/paper/2603.15616/) (2026) - [Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling](https://mlsanity.com/paper/2604.28185/) (2026) - [TextWand: A Unified Framework for Scene Text Editing](https://mlsanity.com/paper/2606.05730/) (2026) - [InnoText: A Unified Model for Visual Text Generation and Editing](https://mlsanity.com/paper/2607.22101/) (2026) - [GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors](https://mlsanity.com/paper/2609.02349/) (2026) - [PosterIQ: A Design Perspective Benchmark for Poster Understanding and Generation](https://mlsanity.com/paper/2603.24078/) (2026) - [Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation](https://mlsanity.com/paper/2609.22916/) (2026) ## Abstract (arXiv) This work demonstrates that diffusion models can achieve font-controllable multilingual text rendering using just raw images without font label annotations.Visual text rendering remains a significant challenge. While recent methods condition diffusion on glyphs, it is impossible to retrieve exact font annotations from large-scale, real-world datasets, which prevents user-specified font control. To address this, we propose a data-driven solution that integrates the conditional diffusion model with a text segmentation model, utilizing segmentation masks to capture and represent fonts in pixel space in a self-supervised manner, thereby eliminating the need for any ground-truth labels and enabling users to customize text rendering with any multilingual font of their choice. The experiment provides a proof of concept of our algorithm in zero-shot text and font editing across diverse fonts and languages, providing valuable insights for the community and industry toward achieving generalized visual text rendering. Code is available at github.com/bowen-upenn/ControlText.