# Candidate Fusion: Integrating Language Modelling into a Sequence-to-Sequence Handwritten Word Recognition Architecture

arXiv: https://arxiv.org/abs/1912.10308 · HTML: https://mlsanity.com/paper/1912.10308/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2019.

## Built on by (most cited first, 12 of 12)

- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Towards the Unseen: Iterative Text Recognition by Distilling from Errors](https://mlsanity.com/paper/2107.12081/) (2021)
- [AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks](https://mlsanity.com/paper/2201.09390/) (2022)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Enhance to Read Better: A Multi-Task Adversarial Network for Handwritten Document Image Enhancement](https://mlsanity.com/paper/2105.12710/) (2021)
- [Evaluation of HTR models without Ground Truth Material](https://mlsanity.com/paper/2201.06170/) (2022)
- [GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System](https://mlsanity.com/paper/2404.14062/) (2024)
- [Proceedings of the 4th International Workshop on Reading Music Systems](https://mlsanity.com/paper/2211.13285/) (2022)
- [CNN-BiLSTM model for English Handwriting Recognition: Comprehensive Evaluation on the IAM Dataset](https://mlsanity.com/paper/2307.00664/) (2023)
- [Preserving Privacy Without Compromising Accuracy: Machine Unlearning for Handwritten Text Recognition](https://mlsanity.com/paper/2504.08616/) (2025)
- [N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition](https://mlsanity.com/paper/2603.03930/) (2026)
- [Learning to Decipher from Pixels: A Case Study of Copiale](https://mlsanity.com/paper/2604.23683/) (2026)

## Abstract (arXiv)

Sequence-to-sequence models have recently become very popular for tackling handwritten word recognition problems. However, how to effectively integrate an external language model into such recognizer is still a challenging problem. The main challenge faced when training a language model is to deal with the language model corpus which is usually different to the one used for training the handwritten word recognition system. Thus, the bias between both word corpora leads to incorrectness on the transcriptions, providing similar or even worse performances on the recognition task. In this work, we introduce Candidate Fusion, a novel way to integrate an external language model to a sequence-to-sequence architecture. Moreover, it provides suggestions from an external language knowledge, as a new input to the sequence-to-sequence recognizer. Hence, Candidate Fusion provides two improvements. On the one hand, the sequence-to-sequence recognizer has the flexibility not only to combine the information from itself and the language model, but also to choose the importance of the information provided by the language model. On the other hand, the external language model has the ability to adapt itself to the training corpus and even learn the most commonly errors produced from the recognizer. Finally, by conducting comprehensive experiments, the Candidate Fusion proves to outperform the state-of-the-art language models for handwritten word recognition tasks.