# Handwriting Transformers

arXiv: https://arxiv.org/abs/2104.03964 · HTML: https://mlsanity.com/paper/2104.03964/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Text and Style Conditioned GAN for Generation of Offline Handwriting Lines](https://mlsanity.com/paper/2009.00678/) (2020)
- [STCN: Stochastic Temporal Convolutional Networks](https://mlsanity.com/paper/1902.06568/) (2019)
- [Generating Handwriting via Decoupled Style Descriptors](https://mlsanity.com/paper/2008.11354/) (2020)
- [Generating Handwritten Chinese Characters using CycleGAN](https://mlsanity.com/paper/1801.08624/) (2018)
- [Spatio-Temporal Handwriting Imitation](https://mlsanity.com/paper/2003.10593/) (2020)

## Built on by (most cited first, 30 of 37)

- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)
- [Disentangling Writer and Character Styles for Handwriting Generation](https://mlsanity.com/paper/2303.14736/) (2023)
- [Few-Shot Font Generation by Learning Fine-Grained Local Styles](https://mlsanity.com/paper/2205.09965/) (2022)
- [Look Closer to Supervise Better: One-Shot Font Generation via Component-Based Discriminator](https://mlsanity.com/paper/2205.00146/) (2022)
- [Handwritten Text Generation from Visual Archetypes](https://mlsanity.com/paper/2303.15269/) (2023)
- [WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models](https://mlsanity.com/paper/2303.16576/) (2023)
- [Object Detection with Transformers: A Review](https://mlsanity.com/paper/2306.04670/) (2023)
- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [VATr++: Choose Your Words Wisely for Handwritten Text Generation](https://mlsanity.com/paper/2402.10798/) (2024)
- [AGTGAN: Unpaired Image Translation for Photographic Ancient Character Generation](https://mlsanity.com/paper/2303.07012/) (2023)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [Style Equalization: Unsupervised Learning of Controllable Generative Sequence Models](https://mlsanity.com/paper/2110.02891/) (2021)
- [Efficient OCR for Building a Diverse Digital History](https://mlsanity.com/paper/2304.02737/) (2023)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [Improving Handwritten OCR with Training Samples Generated by Glyph Conditional Denoising Diffusion Probabilistic Model](https://mlsanity.com/paper/2305.19543/) (2023)
- [EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge](https://mlsanity.com/paper/2310.10050/) (2023)
- [Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models](https://mlsanity.com/paper/2409.00786/) (2024)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)
- [APRNet: Attention-based Pixel-wise Rendering Network for Photo-Realistic Text Image Generation](https://mlsanity.com/paper/2203.07705/) (2022)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation](https://mlsanity.com/paper/2508.17017/) (2025)
- [Data Incubation -- Synthesizing Missing Data for Handwriting Recognition](https://mlsanity.com/paper/2110.07040/) (2021)
- [DELINE8K: A Synthetic Data Pipeline for the Semantic Segmentation of Historical Documents](https://mlsanity.com/paper/2404.19259/) (2024)
- [FW-GAN: Frequency-Driven Handwriting Synthesis with Wave-Modulated MLP Generator](https://mlsanity.com/paper/2508.21040/) (2025)
- [CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantization](https://mlsanity.com/paper/2603.07543/) (2026)
- [WriteViT: Handwritten Text Generation with Vision Transformer](https://mlsanity.com/paper/2505.13235/) (2025)
- [ScriptViT: Vision Transformer-Based Personalized Handwriting Generation](https://mlsanity.com/paper/2511.18307/) (2025)

## Abstract (arXiv)

We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns. The proposed HWT captures the long and short range relationships within the style examples through a self-attention mechanism, thereby encoding both global and local style patterns. Further, the proposed transformer-based HWT comprises an encoder-decoder attention that enables style-content entanglement by gathering the style representation of each query character. To the best of our knowledge, we are the first to introduce a transformer-based generative network for styled handwritten text generation. Our proposed HWT generates realistic styled handwritten text images and significantly outperforms the state-of-the-art demonstrated through extensive qualitative, quantitative and human-based evaluations. The proposed HWT can handle arbitrary length of text and any desired writing style in a few-shot setting. Further, our HWT generalizes well to the challenging scenario where both words and writing style are unseen during training, generating realistic styled handwritten text images.