# Complex Handwriting Trajectory Recovery: Evaluation Metrics and Algorithm

arXiv: https://arxiv.org/abs/2210.15879 · HTML: https://mlsanity.com/paper/2210.15879/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Drawing and Recognizing Chinese Characters with Recurrent Neural Network](https://mlsanity.com/paper/1606.06539/) (2016)
- [Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders](https://mlsanity.com/paper/1906.06142/) (2019)
- [Online trajectory recovery from offline handwritten Japanese kanji characters](https://mlsanity.com/paper/2009.04284/) (2020)
- [Handwriting Trajectory Recovery using End-to-End Deep Encoder-Decoder Network](https://mlsanity.com/paper/1801.07211/) (2018)
- [TRACE: A Differentiable Approach to Line-level Stroke Recovery for Offline Handwritten Text](https://mlsanity.com/paper/2105.11559/) (2021)

## Built on by (most cited first, 9 of 9)

- [Disentangling Writer and Character Styles for Handwriting Generation](https://mlsanity.com/paper/2303.14736/) (2023)
- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write](https://mlsanity.com/paper/2402.05804/) (2024)
- [Mixture-of-experts for handwriting trajectory reconstruction from IMU sensors](https://mlsanity.com/paper/2607.26708/) (2026)
- [DiffInk: Glyph- and Style-Aware Latent Diffusion Transformer for Text to Online Handwriting Generation](https://mlsanity.com/paper/2509.23624/) (2025)
- [Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network](https://mlsanity.com/paper/2607.26733/) (2026)
- [DNA: Dual-branch Network with Adaptation for Open-Set Online Handwriting Generation](https://mlsanity.com/paper/2511.22064/) (2025)
- [Handwriting Trajectory Recovery with Diffusion Models](https://mlsanity.com/paper/2607.03422/) (2026)
- [Handwriting Trajectory Recovery via Autoregressive Ordered Stroke Instance Prediction](https://mlsanity.com/paper/2609.02251/) (2026)

## Abstract (arXiv)

Many important tasks such as forensic signature verification, calligraphy synthesis, etc, rely on handwriting trajectory recovery of which, however, even an appropriate evaluation metric is still missing. Indeed, existing metrics only focus on the writing orders but overlook the fidelity of glyphs. Taking both facets into account, we come up with two new metrics, the adaptive intersection on union (AIoU) which eliminates the influence of various stroke widths, and the length-independent dynamic time warping (LDTW) which solves the trajectory-point alignment problem. After that, we then propose a novel handwriting trajectory recovery model named Parsing-and-tracing ENcoder-decoder Network (PEN-Net), in particular for characters with both complex glyph and long trajectory, which was believed very challenging. In the PEN-Net, a carefully designed double-stream parsing encoder parses the glyph structure, and a global tracing decoder overcomes the memory difficulty of long trajectory prediction. Our experiments demonstrate that the two new metrics AIoU and LDTW together can truly assess the quality of handwriting trajectory recovery and the proposed PEN-Net exhibits satisfactory performance in various complex-glyph languages including Chinese, Japanese and Indic.