# Neural OCR Post-Hoc Correction of Historical Corpora

arXiv: https://arxiv.org/abs/2102.00583 · HTML: https://mlsanity.com/paper/2102.00583/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 1 Feb 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Character-Aware Neural Language Models](https://mlsanity.com/paper/1508.06615/) (2015)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [Character-based Neural Machine Translation](https://mlsanity.com/paper/1603.00810/) (2016)
- [A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation](https://mlsanity.com/paper/1603.06147/) (2016)
- [Neural Language Correction with Character-Based Attention](https://mlsanity.com/paper/1603.09727/) (2016)
- [Incorporating Structural Alignment Biases into an Attentional Neural Translation Model](https://mlsanity.com/paper/1601.01085/) (2016)
- [Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs](https://mlsanity.com/paper/1508.00657/) (2015)
- [Still not there? Comparing Traditional Sequence-to-Sequence Models to Encoder-Decoder Neural Networks on Monotone String Translation Tasks](https://mlsanity.com/paper/1610.07796/) (2016)
- [Improving OCR Accuracy on Early Printed Books by utilizing Cross Fold Training and Voting](https://mlsanity.com/paper/1711.09670/) (2017)
- [State of the Art Optical Character Recognition of 19th Century Fraktur Scripts using Open Source Engines](https://mlsanity.com/paper/1810.03436/) (2018)
- [Character-Level Models versus Morphology in Semantic Role Labeling](https://mlsanity.com/paper/1805.11937/) (2018)

## Built on by (most cited first, 12 of 12)

- [Efficient OCR for Building a Diverse Digital History](https://mlsanity.com/paper/2304.02737/) (2023)
- [Multilingual Event Extraction from Historical Newspaper Adverts](https://mlsanity.com/paper/2305.10928/) (2023)
- [EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge](https://mlsanity.com/paper/2310.10050/) (2023)
- [PHD: Pixel-Based Language Modeling of Historical Documents](https://mlsanity.com/paper/2310.18343/) (2023)
- [Deep Learning for Economists](https://mlsanity.com/paper/2407.15339/) (2024)
- [Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI](https://mlsanity.com/paper/2310.09091/) (2023)
- [RoundTripOCR: A Data Generation Technique for Enhancing Post-OCR Error Correction in Low-Resource Devanagari Languages](https://mlsanity.com/paper/2412.15248/) (2024)
- [Quantifying Character Similarity with Vision Transformers](https://mlsanity.com/paper/2305.14672/) (2023)
- [Optimization of Image Processing Algorithms for Character Recognition in Cultural Typewritten Documents](https://mlsanity.com/paper/2311.15740/) (2023)
- [OCR-EDR: Rendering-Aware Diagnosis and Repair for Closed-Loop OCR Improvement](https://mlsanity.com/paper/2609.03445/) (2026)
- [Low-Resource Heuristics for Bahnaric Optical Character Recognition Improvement](https://mlsanity.com/paper/2601.02965/) (2026)
- [Efficient Domain Adaptation for Text Line Recognition via Decoupled Language Models](https://mlsanity.com/paper/2603.28028/) (2026)

## Abstract (arXiv)

Optical character recognition (OCR) is crucial for a deeper access to historical collections. OCR needs to account for orthographic variations, typefaces, or language evolution (i.e., new letters, word spellings), as the main source of character, word, or word segmentation transcription errors. For digital corpora of historical prints, the errors are further exacerbated due to low scan quality and lack of language standardization. For the task of OCR post-hoc correction, we propose a neural approach based on a combination of recurrent (RNN) and deep convolutional network (ConvNet) to correct OCR transcription errors. At character level we flexibly capture errors, and decode the corrected output based on a novel attention mechanism. Accounting for the input and output similarity, we propose a new loss function that rewards the model's correcting behavior. Evaluation on a historical book corpus in German language shows that our models are robust in capturing diverse OCR transcription errors and reduce the word error rate of 32.3% by more than 89%.