# Utilizing Resource-Rich Language Datasets for End-to-End Scene Text Recognition in Resource-Poor Languages

arXiv: https://arxiv.org/abs/2111.12276 · HTML: https://mlsanity.com/paper/2111.12276/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Robust Scene Text Recognition with Automatic Rectification](https://mlsanity.com/paper/1603.03915/) (2016)
- [Scene Text Detection and Recognition: The Deep Learning Era](https://mlsanity.com/paper/1811.04256/) (2018)
- [Recursive Recurrent Nets with Attention Modeling for OCR in the Wild](https://mlsanity.com/paper/1603.03101/) (2016)
- [Rosetta: Large scale system for text detection and recognition in images](https://mlsanity.com/paper/1910.05085/) (2019)
- [MASTER: Multi-Aspect Non-local Network for Scene Text Recognition](https://mlsanity.com/paper/1910.02562/) (2019)
- [Verisimilar Image Synthesis for Accurate Detection and Recognition of Texts in Scenes](https://mlsanity.com/paper/1807.03021/) (2018)
- [UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World](https://mlsanity.com/paper/2003.10608/) (2020)
- [Massively Multilingual Adversarial Speech Recognition](https://mlsanity.com/paper/1904.02210/) (2019)
- [Single Shot Scene Text Retrieval](https://mlsanity.com/paper/1808.09044/) (2018)
- [Integrating Scene Text and Visual Appearance for Fine-Grained Image Classification](https://mlsanity.com/paper/1704.04613/) (2017)
- [A Holistic Representation Guided Attention Network for Scene Text Recognition](https://mlsanity.com/paper/1904.01375/) (2019)
- [Bidirectional Scene Text Recognition with a Single Decoder](https://mlsanity.com/paper/1912.03656/) (2019)

## Built on by (most cited first, 1 of 1)

- [Extending TrOCR for Text Localization-Free OCR of Full-Page Scanned Receipt Images](https://mlsanity.com/paper/2212.05525/) (2022)

## Abstract (arXiv)

This paper presents a novel training method for end-to-end scene text recognition. End-to-end scene text recognition offers high recognition accuracy, especially when using the encoder-decoder model based on Transformer. To train a highly accurate end-to-end model, we need to prepare a large image-to-text paired dataset for the target language. However, it is difficult to collect this data, especially for resource-poor languages. To overcome this difficulty, our proposed method utilizes well-prepared large datasets in resource-rich languages such as English, to train the resource-poor encoder-decoder model. Our key idea is to build a model in which the encoder reflects knowledge of multiple languages while the decoder specializes in knowledge of just the resource-poor language. To this end, the proposed method pre-trains the encoder by using a multilingual dataset that combines the resource-poor language's dataset and the resource-rich language's dataset to learn language-invariant knowledge for scene text recognition. The proposed method also pre-trains the decoder by using the resource-poor language's dataset to make the decoder better suited to the resource-poor language. Experiments on Japanese scene text recognition using a small, publicly available dataset demonstrate the effectiveness of the proposed method.