# Self-Supervised Representation Learning for Online Handwriting Text Classification

arXiv: https://arxiv.org/abs/2310.06645 · HTML: https://mlsanity.com/paper/2310.06645/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Self-Supervised Feature Learning by Learning to Spot Artifacts](https://mlsanity.com/paper/1806.05024/) (2018)
- [Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting](https://mlsanity.com/paper/2103.13716/) (2021)
- [Fast Multi-language LSTM-based Online Handwriting Recognition](https://mlsanity.com/paper/1902.10525/) (2019)
- [DeepWriterID: An End-to-end Online Text-independent Writer Identification System](https://mlsanity.com/paper/1508.04945/) (2015)
- [Preliminary experiments on automatic gender recognition based on online capital letters](https://mlsanity.com/paper/2203.06265/) (2022)
- [Gender classification by means of online uppercase handwriting: A text-dependent allographic approach](https://mlsanity.com/paper/2203.09848/) (2022)

## Built on by (most cited first, 2 of 2)

- [Self-Supervised Learning Based Handwriting Verification](https://mlsanity.com/paper/2405.18320/) (2024)
- [Contrastive Masked Autoencoders for Character-Level Open-Set Writer Identification](https://mlsanity.com/paper/2501.11895/) (2025)

## Abstract (arXiv)

Self-supervised learning offers an efficient way of extracting rich representations from various types of unlabeled data while avoiding the cost of annotating large-scale datasets. This is achievable by designing a pretext task to form pseudo labels with respect to the modality and domain of the data. Given the evolving applications of online handwritten texts, in this study, we propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages, along with two suggested pipelines for fine-tuning the pretrained models. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification, also highlighting the superiority of utilizing the pretrained models over the models trained from scratch.