# WordArt Designer: User-Driven Artistic Typography Synthesis using Large Language Models

arXiv: https://arxiv.org/abs/2310.18332 · HTML: https://mlsanity.com/paper/2310.18332/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [Composer: Creative and Controllable Image Synthesis with Composable Conditions](https://mlsanity.com/paper/2302.09778/) (2023)
- [GlyphControl: Glyph Conditional Control for Visual Text Generation](https://mlsanity.com/paper/2305.18259/) (2023)
- [Word-As-Image for Semantic Typography](https://mlsanity.com/paper/2303.01818/) (2023)
- [Multi-View Image Generation from a Single-View](https://mlsanity.com/paper/1704.04886/) (2017)
- [Video2Shop: Exact Matching Clothes in Videos to Online Shopping Images](https://mlsanity.com/paper/1804.05287/) (2018)
- [DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion](https://mlsanity.com/paper/2303.09604/) (2023)
- [ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules](https://mlsanity.com/paper/2304.02173/) (2023)
- [Generating Person Images with Appearance-aware Pose Stylizer](https://mlsanity.com/paper/2007.09077/) (2020)

## Built on by (most cited first, 15 of 15)

- [COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design](https://mlsanity.com/paper/2311.16974/) (2023)
- [POSTA: A Go-to Framework for Customized Artistic Poster Generation](https://mlsanity.com/paper/2503.14908/) (2025)
- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [T2VTextBench: A Human Evaluation Benchmark for Textual Control in Video Generation Models](https://mlsanity.com/paper/2505.04946/) (2025)
- [FontStudio: Shape-Adaptive Diffusion Model for Coherent and Consistent Font Effect Generation](https://mlsanity.com/paper/2406.08392/) (2024)
- [Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation](https://mlsanity.com/paper/2407.14774/) (2024)
- [MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis](https://mlsanity.com/paper/2406.19859/) (2024)
- [Typographic Text Generation with Off-the-Shelf Diffusion Model](https://mlsanity.com/paper/2402.14314/) (2024)
- [VitaGlyph: Vitalizing Artistic Typography with Flexible Dual-branch Diffusion Models](https://mlsanity.com/paper/2410.01738/) (2024)
- [GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts](https://mlsanity.com/paper/2411.11435/) (2024)
- [WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope](https://mlsanity.com/paper/2401.01699/) (2024)
- [Khattat: Enhancing Readability and Concept Representation of Semantic Typography](https://mlsanity.com/paper/2410.03748/) (2024)
- [FontCrafter: High-Fidelity Element-Driven Artistic Font Creation with Visual In-Context Generation](https://mlsanity.com/paper/2603.22054/) (2026)
- [Cross-Domain Image Conversion by CycleDM](https://mlsanity.com/paper/2403.02919/) (2024)
- [WordCraft: Interactive Artistic Typography with Attention Awareness and Noise Blending](https://mlsanity.com/paper/2507.09573/) (2025)

## Abstract (arXiv)

This paper introduces WordArt Designer, a user-driven framework for artistic typography synthesis, relying on the Large Language Model (LLM). The system incorporates four key modules: the LLM Engine, SemTypo, StyTypo, and TexTypo modules. 1) The LLM Engine, empowered by the LLM (e.g., GPT-3.5), interprets user inputs and generates actionable prompts for the other modules, thereby transforming abstract concepts into tangible designs. 2) The SemTypo module optimizes font designs using semantic concepts, striking a balance between artistic transformation and readability. 3) Building on the semantic layout provided by the SemTypo module, the StyTypo module creates smooth, refined images. 4) The TexTypo module further enhances the design's aesthetics through texture rendering, enabling the generation of inventive textured fonts. Notably, WordArt Designer highlights the fusion of generative AI with artistic typography. Experience its capabilities on ModelScope: https://www.modelscope.cn/studios/WordArt/WordArt.