# Detecting Handwritten Mathematical Terms with Sensor Based Data

arXiv: https://arxiv.org/abs/2109.05594 · HTML: https://mlsanity.com/paper/2109.05594/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 15 references that are in the map.
- First posted on arXiv 12 Sep 2021.

## Builds on (most cited first, 3 of 3)

- [Bengali Handwritten Character Classification using Transfer Learning on Deep Convolutional Neural Network](https://mlsanity.com/paper/1902.11133/) (2019)
- [Ubicomp Digital 2020 -- Handwriting classification using a convolutional recurrent network](https://mlsanity.com/paper/2008.01078/) (2020)
- [Digitizing Handwriting with a Sensor Pen: A Writer-Independent Recognizer](https://mlsanity.com/paper/2107.03704/) (2021)

## Built on by (most cited first, 3 of 3)

- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Mixture-of-experts for handwriting trajectory reconstruction from IMU sensors](https://mlsanity.com/paper/2607.26708/) (2026)
- [Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network](https://mlsanity.com/paper/2607.26733/) (2026)

## Abstract (arXiv)

In this work we propose a solution to the UbiComp 2021 Challenge by Stabilo in which handwritten mathematical terms are supposed to be automatically classified based on time series sensor data captured on the DigiPen. The input data set contains data of different writers, with label strings constructed from a total of 15 different possible characters. The label should first be split into separate characters to classify them one by one. This issue is solved by applying a data-dependant and rule-based information extraction algorithm to the labeled data. Using the resulting data, two classifiers are constructed. The first is a binary classifier that is able to predict, for unknown data, if a sample is part of a writing activity, and consists of a Deep Neural Network feature extractor in concatenation with a Random Forest that is trained to classify the extracted features at an F1 score of >90%. The second classifier is a Deep Neural Network that combines convolution layers with recurrent layers to predict windows with a single label, out of the 15 possible classes, at an F1 score of >60%. A simulation of the challenge evaluation procedure reports a Levensthein Distance of 8 and shows that the chosen approach still lacks in overall accuracy and real-time applicability.