# Robust Scene Text Recognition with Automatic Rectification

arXiv: https://arxiv.org/abs/1603.03915 · HTML: https://mlsanity.com/paper/1603.03915/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2016.

## Built on by (most cited first, 30 of 121)

- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting](https://mlsanity.com/paper/2105.03620/) (2021)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [Syntax-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2203.01601/) (2022)
- [What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels](https://mlsanity.com/paper/2103.04400/) (2021)
- [From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models](https://mlsanity.com/paper/2403.12027/) (2024)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition](https://mlsanity.com/paper/2208.00438/) (2022)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition](https://mlsanity.com/paper/2109.04145/) (2021)
- [MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding](https://mlsanity.com/paper/2108.06543/) (2021)
- [ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spotting](https://mlsanity.com/paper/2211.10578/) (2022)
- [CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition](https://mlsanity.com/paper/2111.11011/) (2021)
- [Multi-Granularity Prediction for Scene Text Recognition](https://mlsanity.com/paper/2209.03592/) (2022)
- [Optimal Boxes: Boosting End-to-End Scene Text Recognition by Adjusting Annotated Bounding Boxes via Reinforcement Learning](https://mlsanity.com/paper/2207.11934/) (2022)
- [Primitive Representation Learning for Scene Text Recognition](https://mlsanity.com/paper/2105.04286/) (2021)
- [Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species](https://mlsanity.com/paper/2112.06183/) (2021)
- [TPSNet: Reverse Thinking of Thin Plate Splines for Arbitrary Shape Scene Text Representation](https://mlsanity.com/paper/2110.12826/) (2021)
- [A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer](https://mlsanity.com/paper/2112.04888/) (2021)
- [Self-supervised Implicit Glyph Attention for Text Recognition](https://mlsanity.com/paper/2203.03382/) (2022)
- [Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition](https://mlsanity.com/paper/2107.12090/) (2021)
- [On the Cross-dataset Generalization in License Plate Recognition](https://mlsanity.com/paper/2201.00267/) (2022)
- [Visual Text Generation in the Wild](https://mlsanity.com/paper/2407.14138/) (2024)
- [PP-StructureV2: A Stronger Document Analysis System](https://mlsanity.com/paper/2210.05391/) (2022)

## Abstract (arXiv)

Recognizing text in natural images is a challenging task with many unsolved problems. Different from those in documents, words in natural images often possess irregular shapes, which are caused by perspective distortion, curved character placement, etc. We propose RARE (Robust text recognizer with Automatic REctification), a recognition model that is robust to irregular text. RARE is a specially-designed deep neural network, which consists of a Spatial Transformer Network (STN) and a Sequence Recognition Network (SRN). In testing, an image is firstly rectified via a predicted Thin-Plate-Spline (TPS) transformation, into a more "readable" image for the following SRN, which recognizes text through a sequence recognition approach. We show that the model is able to recognize several types of irregular text, including perspective text and curved text. RARE is end-to-end trainable, requiring only images and associated text labels, making it convenient to train and deploy the model in practical systems. State-of-the-art or highly-competitive performance achieved on several benchmarks well demonstrates the effectiveness of the proposed model.