# On Vocabulary Reliance in Scene Text Recognition

arXiv: https://arxiv.org/abs/2005.03959 · HTML: https://mlsanity.com/paper/2005.03959/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2020.

## Built on by (most cited first, 30 of 39)

- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [Revisiting Scene Text Recognition: A Data Perspective](https://mlsanity.com/paper/2307.08723/) (2023)
- [ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spotting](https://mlsanity.com/paper/2211.10578/) (2022)
- [Multi-Granularity Prediction for Scene Text Recognition](https://mlsanity.com/paper/2209.03592/) (2022)
- [Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition](https://mlsanity.com/paper/2112.12916/) (2021)
- [Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition](https://mlsanity.com/paper/2107.12090/) (2021)
- [Levenshtein OCR](https://mlsanity.com/paper/2209.03594/) (2022)
- [Facial Attribute Transformers for Precise and Robust Makeup Transfer](https://mlsanity.com/paper/2104.02894/) (2021)
- [Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation](https://mlsanity.com/paper/2107.12087/) (2021)
- [CLIPTER: Looking at the Bigger Picture in Scene Text Recognition](https://mlsanity.com/paper/2301.07464/) (2023)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Do We Train on Test Data? The Impact of Near-Duplicates on License Plate Recognition](https://mlsanity.com/paper/2304.04653/) (2023)
- [Towards the Unseen: Iterative Text Recognition by Distilling from Errors](https://mlsanity.com/paper/2107.12081/) (2021)
- [TextAdaIN: Paying Attention to Shortcut Learning in Text Recognizers](https://mlsanity.com/paper/2105.03906/) (2021)
- [Out-of-Vocabulary Challenge Report](https://mlsanity.com/paper/2209.06717/) (2022)
- [DNTextSpotter: Arbitrary-Shaped Scene Text Spotting via Improved Denoising Training](https://mlsanity.com/paper/2408.00355/) (2024)
- [Open-set Text Recognition via Character-Context Decoupling](https://mlsanity.com/paper/2204.05535/) (2022)
- [Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition](https://mlsanity.com/paper/2307.13244/) (2023)
- [Relational Contrastive Learning for Scene Text Recognition](https://mlsanity.com/paper/2308.00508/) (2023)
- [Reciprocal Feature Learning via Explicit and Implicit Tasks in Scene Text Recognition](https://mlsanity.com/paper/2105.06229/) (2021)
- [I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognition](https://mlsanity.com/paper/2105.08383/) (2021)
- [Visual-Semantic Transformer for Scene Text Recognition](https://mlsanity.com/paper/2112.00948/) (2021)
- [Improving Scene Text Recognition for Character-Level Long-Tailed Distribution](https://mlsanity.com/paper/2304.08592/) (2023)
- [Self-Supervised Learning for Text Recognition: A Critical Survey](https://mlsanity.com/paper/2407.19889/) (2024)
- [Decoupling Visual-Semantic Feature Learning for Robust Scene Text Recognition](https://mlsanity.com/paper/2111.12351/) (2021)
- [Vision-Language Adaptive Mutual Decoder for OOV-STR](https://mlsanity.com/paper/2209.00859/) (2022)
- [STEP -- Towards Structured Scene-Text Spotting](https://mlsanity.com/paper/2309.02356/) (2023)
- [MX-Font++: Mixture of Heterogeneous Aggregation Experts for Few-shot Font Generation](https://mlsanity.com/paper/2503.02799/) (2025)
- [Perception and Semantic Aware Regularization for Sequential Confidence Calibration](https://mlsanity.com/paper/2305.19498/) (2023)

## Abstract (arXiv)

The pursuit of high performance on public benchmarks has been the driving force for research in scene text recognition, and notable progress has been achieved. However, a close investigation reveals a startling fact that the state-of-the-art methods perform well on images with words within vocabulary but generalize poorly to images with words outside vocabulary. We call this phenomenon "vocabulary reliance". In this paper, we establish an analytical framework to conduct an in-depth study on the problem of vocabulary reliance in scene text recognition. Key findings include: (1) Vocabulary reliance is ubiquitous, i.e., all existing algorithms more or less exhibit such characteristic; (2) Attention-based decoders prove weak in generalizing to words outside vocabulary and segmentation-based decoders perform well in utilizing visual features; (3) Context modeling is highly coupled with the prediction layers. These findings provide new insights and can benefit future research in scene text recognition. Furthermore, we propose a simple yet effective mutual learning strategy to allow models of two families (attention-based and segmentation-based) to learn collaboratively. This remedy alleviates the problem of vocabulary reliance and improves the overall scene text recognition performance.