# HandwritingAgent: Language-Driven Handwriting Synthesis in Scalable Vector Space

arXiv: https://arxiv.org/abs/2606.18788 · HTML: https://mlsanity.com/paper/2606.18788/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [A Learned Representation for Scalable Vector Graphics](https://mlsanity.com/paper/1904.02632/) (2019)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Handwriting Transformers](https://mlsanity.com/paper/2104.03964/) (2021)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [Auto-Encoder Guided GAN for Chinese Calligraphy Synthesis](https://mlsanity.com/paper/1706.08789/) (2017)
- [Diffusion models for Handwriting Generation](https://mlsanity.com/paper/2011.06704/) (2020)
- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [VATr++: Choose Your Words Wisely for Handwritten Text Generation](https://mlsanity.com/paper/2402.10798/) (2024)
- [DiffusionPen: Towards Controlling the Style of Handwritten Text Generation](https://mlsanity.com/paper/2409.06065/) (2024)
- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention](https://mlsanity.com/paper/2106.09432/) (2021)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)

## Abstract (arXiv)

Teaching machines to emulate natural handwriting styles remains an open challenge, as it requires synthesizing stroke sequences that dynamically vary in shape, texture, pressure and script - not only across individuals, but also within a single person's handwriting. Attempts at this challenge have largely explored deep learning methods in both online and offline settings. However, these approaches are often constrained by style-specific architectural choices, heavy reliance on large datasets, high compute costs, and a lack of flexible control over writing styles through natural language. To this end, we introduce HandwritingAgent, a language-driven agent that can synthesize natural handwriting sequences directly in Scalable Vector Graphics (SVG) format with no need for style-specific training. The agent leverages a large reasoning model to geometrically analyse and autoregressively generate target handwritten glyphs as stroke sequences in a discrete grid canvas environment. Generation is conditioned on texts provided in either conversational or non-conversational mode, along with a reference handwriting-style image. Experiments on diverse handwriting tasks spanning imitation, recognition, multi-lingual handwriting synthesis, and generation of complex handwritten maths and science expressions indicate substantial improvement in performance, with HandwritingAgent matching or surpassing state-of-the-art generative handwriting models, while providing a more efficient, controllable, and generalizable synthesis method.