# Dropout improves Recurrent Neural Networks for Handwriting Recognition

arXiv: https://arxiv.org/abs/1312.4569 · HTML: https://mlsanity.com/paper/1312.4569/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2013.

## Built on by (most cited first, 30 of 39)

- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [On the statistical complexity of quantum circuits](https://mlsanity.com/paper/2101.06154/) (2021)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition](https://mlsanity.com/paper/2208.07682/) (2022)
- [Protein Language Models and Structure Prediction: Connection and Progression](https://mlsanity.com/paper/2211.16742/) (2022)
- [Best Practices for a Handwritten Text Recognition System](https://mlsanity.com/paper/2404.11339/) (2024)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens](https://mlsanity.com/paper/2202.07036/) (2022)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [A Transformer-based Approach for Arabic Offline Handwritten Text Recognition](https://mlsanity.com/paper/2307.15045/) (2023)
- [Batch Normalization Explained](https://mlsanity.com/paper/2209.14778/) (2022)
- [Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition](https://mlsanity.com/paper/2407.13559/) (2024)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Easter2.0: Improving convolutional models for handwritten text recognition](https://mlsanity.com/paper/2205.14879/) (2022)
- [Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph](https://mlsanity.com/paper/2501.00659/) (2024)
- [A Survey on Dropout Methods and Experimental Verification in Recommendation](https://mlsanity.com/paper/2204.02027/) (2022)
- [Large language models for newspaper sentiment analysis during COVID-19: The Guardian](https://mlsanity.com/paper/2405.13056/) (2024)
- [General Detection-based Text Line Recognition](https://mlsanity.com/paper/2409.17095/) (2024)
- [Fine-tuning Handwriting Recognition systems with Temporal Dropout](https://mlsanity.com/paper/2102.00511/) (2021)
- [Calibrating the Lee-Carter and the Poisson Lee-Carter models via Neural Networks](https://mlsanity.com/paper/2106.12312/) (2021)
- [Snapture -- A Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2205.15862/) (2022)
- [Information Geometry of Dropout Training](https://mlsanity.com/paper/2206.10936/) (2022)
- [2D Self-Organized ONN Model For Handwritten Text Recognition](https://mlsanity.com/paper/2207.08139/) (2022)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [A Lexicon and Depth-wise Separable Convolution Based Handwritten Text Recognition System](https://mlsanity.com/paper/2207.04651/) (2022)
- [An Empirical Study of AI Techniques in Mobile Applications](https://mlsanity.com/paper/2212.01635/) (2022)
- [GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System](https://mlsanity.com/paper/2404.14062/) (2024)
- [DRetHTR: Linear-Time Decoder-Only Retentive Network for Handwritten Text Recognition](https://mlsanity.com/paper/2602.17387/) (2026)

## Abstract (arXiv)

Recurrent neural networks (RNNs) with Long Short-Term memory cells currently hold the best known results in unconstrained handwriting recognition. We show that their performance can be greatly improved using dropout - a recently proposed regularization method for deep architectures. While previous works showed that dropout gave superior performance in the context of convolutional networks, it had never been applied to RNNs. In our approach, dropout is carefully used in the network so that it does not affect the recurrent connections, hence the power of RNNs in modeling sequence is preserved. Extensive experiments on a broad range of handwritten databases confirm the effectiveness of dropout on deep architectures even when the network mainly consists of recurrent and shared connections.