# What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels

arXiv: https://arxiv.org/abs/2103.04400 · HTML: https://mlsanity.com/paper/2103.04400/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 78 references that are in the map.
- First posted on arXiv 7 Mar 2021.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)

## Built on by (most cited first, 30 of 63)

- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System](https://mlsanity.com/paper/2206.03001/) (2022)
- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [Revisiting Scene Text Recognition: A Data Perspective](https://mlsanity.com/paper/2307.08723/) (2023)
- [DTrOCR: Decoder-only Transformer for Optical Character Recognition](https://mlsanity.com/paper/2308.15996/) (2023)
- [CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition](https://mlsanity.com/paper/2111.11011/) (2021)
- [CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model](https://mlsanity.com/paper/2305.14014/) (2023)
- [Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting](https://mlsanity.com/paper/2203.03911/) (2022)
- [Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition](https://mlsanity.com/paper/2112.12916/) (2021)
- [Self-supervised Implicit Glyph Attention for Text Recognition](https://mlsanity.com/paper/2203.03382/) (2022)
- [Visual Text Generation in the Wild](https://mlsanity.com/paper/2407.14138/) (2024)
- [SimAN: Exploring Self-Supervised Representation Learning of Scene Text via Similarity-Aware Normalization](https://mlsanity.com/paper/2203.10492/) (2022)
- [Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition](https://mlsanity.com/paper/2303.04291/) (2023)
- [Multimodal Semi-Supervised Learning for Text Recognition](https://mlsanity.com/paper/2205.03873/) (2022)
- [CLIPTER: Looking at the Bigger Picture in Scene Text Recognition](https://mlsanity.com/paper/2301.07464/) (2023)
- [Scene Text Image Super-resolution based on Text-conditional Diffusion Models](https://mlsanity.com/paper/2311.09759/) (2023)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition](https://mlsanity.com/paper/2310.04999/) (2023)
- [COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts](https://mlsanity.com/paper/2207.04675/) (2022)
- [LISTER: Neighbor Decoding for Length-Insensitive Scene Text Recognition](https://mlsanity.com/paper/2308.12774/) (2023)
- [SGBANet: Semantic GAN and Balanced Attention Network for Arbitrarily Oriented Scene Text Recognition](https://mlsanity.com/paper/2207.10256/) (2022)
- [Pure Transformer with Integrated Experts for Scene Text Recognition](https://mlsanity.com/paper/2211.04963/) (2022)
- [Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach](https://mlsanity.com/paper/2308.08806/) (2023)
- [TextSSR: Diffusion-based Data Synthesis for Scene Text Recognition](https://mlsanity.com/paper/2412.01137/) (2024)
- [Pushing the Performance Limit of Scene Text Recognizer without Human Annotation](https://mlsanity.com/paper/2204.07714/) (2022)
- [DiffusionSTR: Diffusion Model for Scene Text Recognition](https://mlsanity.com/paper/2306.16707/) (2023)
- [Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition](https://mlsanity.com/paper/2307.13244/) (2023)
- [An Empirical Study of Scaling Law for OCR](https://mlsanity.com/paper/2401.00028/) (2023)

## Abstract (arXiv)

Scene text recognition (STR) task has a common practice: All state-of-the-art STR models are trained on large synthetic data. In contrast to this practice, training STR models only on fewer real labels (STR with fewer labels) is important when we have to train STR models without synthetic data: for handwritten or artistic texts that are difficult to generate synthetically and for languages other than English for which we do not always have synthetic data. However, there has been implicit common knowledge that training STR models on real data is nearly impossible because real data is insufficient. We consider that this common knowledge has obstructed the study of STR with fewer labels. In this work, we would like to reactivate STR with fewer labels by disproving the common knowledge. We consolidate recently accumulated public real data and show that we can train STR models satisfactorily only with real labeled data. Subsequently, we find simple data augmentation to fully exploit real data. Furthermore, we improve the models by collecting unlabeled data and introducing semi- and self-supervised methods. As a result, we obtain a competitive model to state-of-the-art methods. To the best of our knowledge, this is the first study that 1) shows sufficient performance by only using real labels and 2) introduces semi- and self-supervised methods into STR with fewer labels. Our code and data are available: https://github.com/ku21fan/STR-Fewer-Labels