# Character-Based Handwritten Text Transcription with Attention Networks

arXiv: https://arxiv.org/abs/1712.04046 · HTML: https://mlsanity.com/paper/1712.04046/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2017.

## Built on by (most cited first, 11 of 11)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Lexicon and Attention based Handwritten Text Recognition System](https://mlsanity.com/paper/2209.04817/) (2022)
- [Online Gesture Recognition using Transformer and Natural Language Processing](https://mlsanity.com/paper/2305.03407/) (2023)
- [Spatial Context-based Self-Supervised Learning for Handwritten Text Recognition](https://mlsanity.com/paper/2404.11585/) (2024)
- [Deep Generative Models for Vehicle Speed Trajectories](https://mlsanity.com/paper/2112.08361/) (2021)
- [A Transformer Architecture for Online Gesture Recognition of Mathematical Expressions](https://mlsanity.com/paper/2211.02643/) (2022)
- [Understanding attention-based encoder-decoder networks: a case study with chess scoresheet recognition](https://mlsanity.com/paper/2406.06538/) (2024)
- [Boosting CNN-based Handwriting Recognition Systems with Learnable Relaxation Labeling](https://mlsanity.com/paper/2409.05699/) (2024)
- [Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies](https://mlsanity.com/paper/2604.09713/) (2026)

## Abstract (arXiv)

The paper approaches the task of handwritten text recognition (HTR) with attentional encoder-decoder networks trained on sequences of characters, rather than words. We experiment on lines of text from popular handwriting datasets and compare different activation functions for the attention mechanism used for aligning image pixels and target characters. We find that softmax attention focuses heavily on individual characters, while sigmoid attention focuses on multiple characters at each step of the decoding. When the sequence alignment is one-to-one, softmax attention is able to learn a more precise alignment at each step of the decoding, whereas the alignment generated by sigmoid attention is much less precise. When a linear function is used to obtain attention weights, the model predicts a character by looking at the entire sequence of characters and performs poorly because it lacks a precise alignment between the source and target. Future research may explore HTR in natural scene images, since the model is capable of transcribing handwritten text without the need for producing segmentations or bounding boxes of text in images.