# Towards an IMU-based Pen Online Handwriting Recognizer

arXiv: https://arxiv.org/abs/2105.12434 · HTML: https://mlsanity.com/paper/2105.12434/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 51 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 3 of 3)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fast Multi-language LSTM-based Online Handwriting Recognition](https://mlsanity.com/paper/1902.10525/) (2019)

## Built on by (most cited first, 5 of 5)

- [Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens](https://mlsanity.com/paper/2202.07036/) (2022)
- [Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead](https://mlsanity.com/paper/2602.07049/) (2026)
- [Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network](https://mlsanity.com/paper/2607.26733/) (2026)
- [Robust and Efficient Writer-Independent IMU-Based Handwriting Recognition](https://mlsanity.com/paper/2502.20954/) (2025)
- [Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition](https://mlsanity.com/paper/2603.16883/) (2026)

## Abstract (arXiv)

Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a online handwriting recognition system for word recognition which is based on inertial measurement units (IMUs) for digitizing text written on paper. This is obtained by means of a sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional and bidirectional LSTM networks, and is trained with the Connectionist Temporal Classification loss that allows the interpretation of raw sensor data into words without the need of sequence segmentation. We use a dataset of words collected using multiple sensor-enhanced pens and evaluate our model on distinct test sets of seen and unseen words achieving a character error rate of 17.97% and 17.08%, respectively, without the use of a dictionary or language model