# Character Queries: A Transformer-based Approach to On-Line Handwritten Character Segmentation

arXiv: https://arxiv.org/abs/2309.03072 · HTML: https://mlsanity.com/paper/2309.03072/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)
- [Generating Handwriting via Decoupled Style Descriptors](https://mlsanity.com/paper/2008.11354/) (2020)
- [Fast Multi-language LSTM-based Online Handwriting Recognition](https://mlsanity.com/paper/1902.10525/) (2019)

## Built on by (most cited first, 3 of 3)

- [Contrastive Masked Autoencoders for Character-Level Open-Set Writer Identification](https://mlsanity.com/paper/2501.11895/) (2025)
- [A Transformer Based Handwriting Recognition System Jointly Using Online and Offline Features](https://mlsanity.com/paper/2506.20255/) (2025)
- [BoundInk: Boundary-Aware Online Handwriting Generation](https://mlsanity.com/paper/2604.02103/) (2026)

## Abstract (arXiv)

On-line handwritten character segmentation is often associated with handwriting recognition and even though recognition models include mechanisms to locate relevant positions during the recognition process, it is typically insufficient to produce a precise segmentation. Decoupling the segmentation from the recognition unlocks the potential to further utilize the result of the recognition. We specifically focus on the scenario where the transcription is known beforehand, in which case the character segmentation becomes an assignment problem between sampling points of the stylus trajectory and characters in the text. Inspired by the k-means clustering algorithm, we view it from the perspective of cluster assignment and present a Transformer-based architecture where each cluster is formed based on a learned character query in the Transformer decoder block. In order to assess the quality of our approach, we create character segmentation ground truths for two popular on-line handwriting datasets, IAM-OnDB and HANDS-VNOnDB, and evaluate multiple methods on them, demonstrating that our approach achieves the overall best results.