# Handwriting Trajectory Recovery with Diffusion Models

arXiv: https://arxiv.org/abs/2607.03422 · HTML: https://mlsanity.com/paper/2607.03422/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Complex Handwriting Trajectory Recovery: Evaluation Metrics and Algorithm](https://mlsanity.com/paper/2210.15879/) (2022)
- [Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders](https://mlsanity.com/paper/1906.06142/) (2019)
- [Online trajectory recovery from offline handwritten Japanese kanji characters](https://mlsanity.com/paper/2009.04284/) (2020)
- [Handwriting Trajectory Recovery using End-to-End Deep Encoder-Decoder Network](https://mlsanity.com/paper/1801.07211/) (2018)
- [Writing Order Recovery in Complex and Long Static Handwriting](https://mlsanity.com/paper/2406.03194/) (2024)

## Abstract (arXiv)

Recovering online pen trajectories from offline handwriting images, often referred to as handwriting trajectory recovery (stroke recovery), is an offline-to-online conversion task with applications in stroke-level editing and forensic analysis. We propose, to the best of our knowledge, the first diffusion-model-based framework for this task. Our method formulates trajectory recovery as image-conditioned generation and uses a denoising diffusion model to sample pen trajectories consistent with the observed ink trace. Through extensive quantitative evaluations on CASIA-OLHWDB (1.0-1.1), we verify that the proposed approach enables accurate recovery even for complex multi-stroke characters, substantially improving both temporal similarity (DTW/LDTW) and shape fidelity (AIoU) over representative prior methods such as PEN-Net and Cross-VAE. We further show that the model captures general stroke-order tendencies and generalizes to classes unseen during training, exemplified by cross-script transfer: a model trained on Chinese characters can recover reasonable stroke orders for Latin letters to some extent.