# Scene Text Recognition via Transformer

arXiv: https://arxiv.org/abs/2003.08077 · HTML: https://mlsanity.com/paper/2003.08077/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2020.

## Built on by (most cited first, 7 of 7)

- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tables](https://mlsanity.com/paper/2105.06400/) (2021)
- [ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX](https://mlsanity.com/paper/2105.14426/) (2021)
- [Open-set Text Recognition via Character-Context Decoupling](https://mlsanity.com/paper/2204.05535/) (2022)
- [Tables to LaTeX: structure and content extraction from scientific tables](https://mlsanity.com/paper/2210.17246/) (2022)
- [UIT-HWDB: Using Transferring Method to Construct A Novel Benchmark for Evaluating Unconstrained Handwriting Image Recognition in Vietnamese](https://mlsanity.com/paper/2211.05407/) (2022)
- [Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition](https://mlsanity.com/paper/2302.14261/) (2023)

## Abstract (arXiv)

Scene text recognition with arbitrary shape is very challenging due to large variations in text shapes, fonts, colors, backgrounds, etc. Most state-of-the-art algorithms rectify the input image into the normalized image, then treat the recognition as a sequence prediction task. The bottleneck of such methods is the rectification, which will cause errors due to distortion perspective. In this paper, we find that the rectification is completely unnecessary. What all we need is the spatial attention. We therefore propose a simple but extremely effective scene text recognition method based on transformer [50]. Different from previous transformer based models [56,34], which just use the decoder of the transformer to decode the convolutional attention, the proposed method use a convolutional feature maps as word embedding input into transformer. In such a way, our method is able to make full use of the powerful attention mechanism of the transformer. Extensive experimental results show that the proposed method significantly outperforms state-of-the-art methods by a very large margin on both regular and irregular text datasets. On one of the most challenging CUTE dataset whose state-of-the-art prediction accuracy is 89.6%, our method achieves 99.3%, which is a pretty surprising result. We will release our source code and believe that our method will be a new benchmark of scene text recognition with arbitrary shapes.