# Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

arXiv: https://arxiv.org/abs/1610.07844 · HTML: https://mlsanity.com/paper/1610.07844/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2016.

## Built on by (most cited first, 8 of 8)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluation](https://mlsanity.com/paper/2306.12916/) (2023)
- [Lemmatization of Historical Old Literary Finnish Texts in Modern Orthography](https://mlsanity.com/paper/2107.03266/) (2021)
- [Multilingual Event Extraction from Historical Newspaper Adverts](https://mlsanity.com/paper/2305.10928/) (2023)
- [Computational Morphology with Neural Network Approaches](https://mlsanity.com/paper/2105.09404/) (2021)
- [Recent advancements in computational morphology : A comprehensive survey](https://mlsanity.com/paper/2406.05424/) (2024)
- [Two Approaches to Diachronic Normalization of Polish Texts](https://mlsanity.com/paper/2402.01300/) (2024)
- [Two Spelling Normalization Approaches Based on Large Language Models](https://mlsanity.com/paper/2506.23288/) (2025)

## Abstract (arXiv)

Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.