# Towards Open-Set Text Recognition via Label-to-Prototype Learning

arXiv: https://arxiv.org/abs/2203.05179 · HTML: https://mlsanity.com/paper/2203.05179/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 64 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [Few-Shot Image Recognition by Predicting Parameters from Activations](https://mlsanity.com/paper/1706.03466/) (2017)
- [ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT](https://mlsanity.com/paper/1909.07741/) (2019)
- [Robust Scene Text Recognition with Automatic Rectification](https://mlsanity.com/paper/1603.03915/) (2016)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)](https://mlsanity.com/paper/1708.09585/) (2017)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [ESIR: End-to-end Scene Text Recognition via Iterative Image Rectification](https://mlsanity.com/paper/1812.05824/) (2018)
- [NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition](https://mlsanity.com/paper/1806.00926/) (2018)
- [Generating Classification Weights with GNN Denoising Autoencoders for Few-Shot Learning](https://mlsanity.com/paper/1905.01102/) (2019)
- [Rosetta: Large scale system for text detection and recognition in images](https://mlsanity.com/paper/1910.05085/) (2019)
- [SCATTER: Selective Context Attentional Scene Text Recognizer](https://mlsanity.com/paper/2003.11288/) (2020)
- [TextScanner: Reading Characters in Order for Robust Scene Text Recognition](https://mlsanity.com/paper/1912.12422/) (2019)
- [On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention](https://mlsanity.com/paper/1910.04396/) (2019)
- [Scene Text Recognition from Two-Dimensional Perspective](https://mlsanity.com/paper/1809.06508/) (2018)

## Built on by (most cited first, 12 of 12)

- [Handling Out-of-Distribution Data: A Survey](https://mlsanity.com/paper/2507.21160/) (2025)
- [Zero-shot Generation of Training Data with Denoising Diffusion Probabilistic Model for Handwritten Chinese Character Recognition](https://mlsanity.com/paper/2305.15660/) (2023)
- [Class-Aware Mask-Guided Feature Refinement for Scene Text Recognition](https://mlsanity.com/paper/2402.13643/) (2024)
- [Open-set Text Recognition via Character-Context Decoupling](https://mlsanity.com/paper/2204.05535/) (2022)
- [Exploring Diverse Representations for Open Set Recognition](https://mlsanity.com/paper/2401.06521/) (2024)
- [DSText V2: A Comprehensive Video Text Spotting Dataset for Dense and Small Text](https://mlsanity.com/paper/2312.01938/) (2023)
- [HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition](https://mlsanity.com/paper/2403.13761/) (2024)
- [Managing the unknown: a survey on Open Set Recognition and tangential areas](https://mlsanity.com/paper/2312.08785/) (2023)
- [MegaHan97K: A Large-Scale Dataset for Mega-Category Chinese Character Recognition with over 97K Categories](https://mlsanity.com/paper/2506.04807/) (2025)
- [Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction](https://mlsanity.com/paper/2307.16253/) (2023)
- [Word length-aware text spotting: Enhancing detection and recognition in dense text image](https://mlsanity.com/paper/2312.15690/) (2023)
- [MOoSE: Multi-Orientation Sharing Experts for Open-set Scene Text Recognition](https://mlsanity.com/paper/2407.18616/) (2024)

## Abstract (arXiv)

Scene text recognition is a popular topic and extensively used in the industry. Although many methods have achieved satisfactory performance for the close-set text recognition challenges, these methods lose feasibility in open-set scenarios, where collecting data or retraining models for novel characters could yield a high cost. For example, annotating samples for foreign languages can be expensive, whereas retraining the model each time when a novel character is discovered from historical documents costs both time and resources. In this paper, we introduce and formulate a new open-set text recognition task which demands the capability to spot and recognize novel characters without retraining. A label-to-prototype learning framework is also proposed as a baseline for the proposed task. Specifically, the framework introduces a generalizable label-to-prototype mapping function to build prototypes (class centers) for both seen and unseen classes. An open-set predictor is then utilized to recognize or reject samples according to the prototypes. The implementation of rejection capability over out-of-set characters allows automatic spotting of unknown characters in the incoming data stream. Extensive experiments show that our method achieves promising performance on a variety of zero-shot, close-set, and open-set text recognition datasets