# Disentangling Writer and Character Styles for Handwriting Generation

arXiv: https://arxiv.org/abs/2303.14736 · HTML: https://mlsanity.com/paper/2303.14736/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 66 references that are in the map.
- First posted on arXiv 26 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation](https://mlsanity.com/paper/2106.15326/) (2021)
- [Multi-Content GAN for Few-Shot Font Style Transfer](https://mlsanity.com/paper/1712.00516/) (2017)
- [Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition](https://mlsanity.com/paper/2107.09249/) (2021)
- [Sketchformer: Transformer-based Representation for Sketched Structure](https://mlsanity.com/paper/2002.10381/) (2020)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [DG-Font: Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2104.03064/) (2021)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Separating Style and Content for Generalized Style Transfer](https://mlsanity.com/paper/1711.06454/) (2017)
- [Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning](https://mlsanity.com/paper/1910.04987/) (2019)
- [Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning](https://mlsanity.com/paper/2102.06605/) (2021)
- [Few-shot Font Generation with Localized Style Representations and Factorization](https://mlsanity.com/paper/2009.11042/) (2020)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)
- [Handwriting Transformers](https://mlsanity.com/paper/2104.03964/) (2021)
- [Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View](https://mlsanity.com/paper/2102.10543/) (2021)
- [Look Closer to Supervise Better: One-Shot Font Generation via Component-Based Discriminator](https://mlsanity.com/paper/2205.00146/) (2022)
- [Drawing and Recognizing Chinese Characters with Recurrent Neural Network](https://mlsanity.com/paper/1606.06539/) (2016)
- [CoSE: Compositional Stroke Embeddings](https://mlsanity.com/paper/2006.09930/) (2020)

## Built on by (most cited first, 30 of 35)

- [Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collections](https://mlsanity.com/paper/2307.08093/) (2023)
- [DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding](https://mlsanity.com/paper/2408.15045/) (2024)
- [Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning](https://mlsanity.com/paper/2409.01128/) (2024)
- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [HiPA: Enabling One-Step Text-to-Image Diffusion Models via High-Frequency-Promoting Adaptation](https://mlsanity.com/paper/2311.18158/) (2023)
- [KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embedding](https://mlsanity.com/paper/2409.01113/) (2024)
- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [Monocular and Generalizable Gaussian Talking Head Animation](https://mlsanity.com/paper/2504.00665/) (2025)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [ReplayCAD: Generative Diffusion Replay for Continual Anomaly Detection](https://mlsanity.com/paper/2505.06603/) (2025)
- [Decoupling Layout from Glyph in Online Chinese Handwriting Generation](https://mlsanity.com/paper/2410.02309/) (2024)
- [HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding](https://mlsanity.com/paper/2602.12957/) (2026)
- [Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models](https://mlsanity.com/paper/2409.00786/) (2024)
- [Globally Correlation-Aware Hard Negative Generation](https://mlsanity.com/paper/2411.13145/) (2024)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [DeepIcon: A Hierarchical Network for Layer-wise Icon Vectorization](https://mlsanity.com/paper/2410.15760/) (2024)
- [Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions](https://mlsanity.com/paper/2411.03755/) (2024)
- [Unsupervised Learning for Class Distribution Mismatch](https://mlsanity.com/paper/2505.06948/) (2025)
- [Order-Level Attention Similarity Across Language Models: A Latent Commonality](https://mlsanity.com/paper/2511.05064/) (2025)
- [Perception and Semantic Aware Regularization for Sequential Confidence Calibration](https://mlsanity.com/paper/2305.19498/) (2023)
- [DiffInk: Glyph- and Style-Aware Latent Diffusion Transformer for Text to Online Handwriting Generation](https://mlsanity.com/paper/2509.23624/) (2025)
- [OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis](https://mlsanity.com/paper/2603.19613/) (2026)
- [Layout Stroke Imitation: A Layout Guided Handwriting Stroke Generation for Style Imitation with Diffusion Model](https://mlsanity.com/paper/2509.15678/) (2025)
- [DNA: Dual-branch Network with Adaptation for Open-Set Online Handwriting Generation](https://mlsanity.com/paper/2511.22064/) (2025)
- [Telling Human and Machine Handwriting Apart](https://mlsanity.com/paper/2601.11700/) (2026)
- [DohaScript: A Large-Scale Multi-Writer Dataset for Continuous Handwritten Hindi Text](https://mlsanity.com/paper/2602.18089/) (2026)
- [ScribeTokens: Fixed-Vocabulary Tokenization of Digital Ink](https://mlsanity.com/paper/2603.02805/) (2026)
- [One step further with Monte-Carlo sampler to guide diffusion better](https://mlsanity.com/paper/2603.06685/) (2026)
- [BoundInk: Boundary-Aware Online Handwriting Generation](https://mlsanity.com/paper/2604.02103/) (2026)

## Abstract (arXiv)

Training machines to synthesize diverse handwritings is an intriguing task. Recently, RNN-based methods have been proposed to generate stylized online Chinese characters. However, these methods mainly focus on capturing a person's overall writing style, neglecting subtle style inconsistencies between characters written by the same person. For example, while a person's handwriting typically exhibits general uniformity (e.g., glyph slant and aspect ratios), there are still small style variations in finer details (e.g., stroke length and curvature) of characters. In light of this, we propose to disentangle the style representations at both writer and character levels from individual handwritings to synthesize realistic stylized online handwritten characters. Specifically, we present the style-disentangled Transformer (SDT), which employs two complementary contrastive objectives to extract the style commonalities of reference samples and capture the detailed style patterns of each sample, respectively. Extensive experiments on various language scripts demonstrate the effectiveness of SDT. Notably, our empirical findings reveal that the two learned style representations provide information at different frequency magnitudes, underscoring the importance of separate style extraction. Our source code is public at: https://github.com/dailenson/SDT.