# WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models

arXiv: https://arxiv.org/abs/2303.16576 · HTML: https://mlsanity.com/paper/2303.16576/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Symbolic Music Generation with Diffusion Models](https://mlsanity.com/paper/2103.16091/) (2021)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [Handwriting Transformers](https://mlsanity.com/paper/2104.03964/) (2021)
- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Unsupervised Feature Learning for Writer Identification and Writer Retrieval](https://mlsanity.com/paper/1705.09369/) (2017)
- [A Survey of Historical Document Image Datasets](https://mlsanity.com/paper/2203.08504/) (2022)
- [SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators](https://mlsanity.com/paper/2105.10528/) (2021)
- [Encoding CNN Activations for Writer Recognition](https://mlsanity.com/paper/1712.07923/) (2017)
- [Generating Synthetic Handwritten Historical Documents With OCR Constrained GANs](https://mlsanity.com/paper/2103.08236/) (2021)

## Built on by (most cited first, 23 of 23)

- [FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning](https://mlsanity.com/paper/2312.12142/) (2023)
- [Visual Text Generation in the Wild](https://mlsanity.com/paper/2407.14138/) (2024)
- [Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2305.04044/) (2023)
- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [VATr++: Choose Your Words Wisely for Handwritten Text Generation](https://mlsanity.com/paper/2402.10798/) (2024)
- [DiffusionPen: Towards Controlling the Style of Handwritten Text Generation](https://mlsanity.com/paper/2409.06065/) (2024)
- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [Rethinking HTG Evaluation: Bridging Generation and Recognition](https://mlsanity.com/paper/2409.02683/) (2024)
- [Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models](https://mlsanity.com/paper/2409.00786/) (2024)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [StylusAI: Stylistic Adaptation for Robust German Handwritten Text Generation](https://mlsanity.com/paper/2407.15608/) (2024)
- [Latent Diffusion for Guided Document Table Generation](https://mlsanity.com/paper/2408.09800/) (2024)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation](https://mlsanity.com/paper/2508.17017/) (2025)
- [FW-GAN: Frequency-Driven Handwriting Synthesis with Wave-Modulated MLP Generator](https://mlsanity.com/paper/2508.21040/) (2025)
- [CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantization](https://mlsanity.com/paper/2603.07543/) (2026)
- [Few-shot Writer Adaptation via Multimodal In-Context Learning](https://mlsanity.com/paper/2603.29450/) (2026)
- [Enhancing Scene Text Detectors with Realistic Text Image Synthesis Using Diffusion Models](https://mlsanity.com/paper/2311.16555/) (2023)
- [Layout Stroke Imitation: A Layout Guided Handwriting Stroke Generation for Style Imitation with Diffusion Model](https://mlsanity.com/paper/2509.15678/) (2025)
- [ScriptViT: Vision Transformer-Based Personalized Handwriting Generation](https://mlsanity.com/paper/2511.18307/) (2025)
- [Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition](https://mlsanity.com/paper/2603.16883/) (2026)
- [Diffusion-Based Ukrainian Handwritten Text Generation with Cross-Domain Style Transfer](https://mlsanity.com/paper/2605.27487/) (2026)

## Abstract (arXiv)

Text-to-Image synthesis is the task of generating an image according to a specific text description. Generative Adversarial Networks have been considered the standard method for image synthesis virtually since their introduction. Denoising Diffusion Probabilistic Models are recently setting a new baseline, with remarkable results in Text-to-Image synthesis, among other fields. Aside its usefulness per se, it can also be particularly relevant as a tool for data augmentation to aid training models for other document image processing tasks. In this work, we present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles, by using class index styles and text content prompts without the need of adversarial training, writer recognition, or text recognition. We gauge system performance with the Fr'echet Inception Distance, writer recognition accuracy, and writer retrieval. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data. Code is available at: https://github.com/koninik/WordStylist.