# Handwriting Trajectory Recovery via Autoregressive Ordered Stroke Instance Prediction

arXiv: https://arxiv.org/abs/2609.02251 · HTML: https://mlsanity.com/paper/2609.02251/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models](https://mlsanity.com/paper/2606.03748/) (2026)
- [Joint Semantic Segmentation and Boundary Detection using Iterative Pyramid Contexts](https://mlsanity.com/paper/2004.07684/) (2020)
- [Complex Handwriting Trajectory Recovery: Evaluation Metrics and Algorithm](https://mlsanity.com/paper/2210.15879/) (2022)
- [Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders](https://mlsanity.com/paper/1906.06142/) (2019)
- [Online trajectory recovery from offline handwritten Japanese kanji characters](https://mlsanity.com/paper/2009.04284/) (2020)
- [Handwriting Trajectory Recovery using End-to-End Deep Encoder-Decoder Network](https://mlsanity.com/paper/1801.07211/) (2018)
- [Instance Segmentation for Chinese Character Stroke Extraction, Datasets and Benchmarks](https://mlsanity.com/paper/2210.13826/) (2022)
- [InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write](https://mlsanity.com/paper/2402.05804/) (2024)
- [Joint Object Contour Points and Semantics for Instance Segmentation](https://mlsanity.com/paper/2008.00460/) (2020)

## Abstract (arXiv)

Handwriting trajectory recovery aims to infer the dynamic writing process hidden behind a static handwritten image. Since offline handwriting preserves only the final spatial ink pattern, temporal information such as stroke order, writing direction, and pen-tip motion is lost, making recovery inherently ambiguous. Existing learning-based methods often directly predict the complete character trajectory without explicitly exploiting the stroke-level organization of handwriting. We argue that recovering the writing process should follow the writing process itself. Accordingly, we propose a two-stage framework that first recovers ordered stroke instances and then reconstructs continuous within-stroke motion. The first stage integrates stroke extraction and stroke-order recovery through autoregressive ordered stroke prediction, while direction-related structural cues further support within-stroke trajectory generation. Experiments on Chinese handwriting show that the proposed ordered prediction is more effective than post-hoc stroke ordering. Even without trajectory simplification, our full-point model achieves numerically better results than those reported by all compared baselines, while a controlled analysis shows that trajectory sampling density substantially affects measured recovery performance. Additional experiments demonstrate generalization to unseen Chinese character categories and cross-language extensibility to English and Tamil handwriting.