# One Model for Two Tasks: Cooperatively Recognizing and Recovering Low-Resolution Scene Text Images by Iterative Mutual Guidance

arXiv: https://arxiv.org/abs/2409.14483 · HTML: https://mlsanity.com/paper/2409.14483/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 22 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [Learning a No-Reference Quality Metric for Single-Image Super-Resolution](https://mlsanity.com/paper/1612.05890/) (2016)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Focusing Attention: Towards Accurate Text Recognition in Natural Images](https://mlsanity.com/paper/1709.02054/) (2017)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition](https://mlsanity.com/paper/1806.00926/) (2018)
- [Scene Text Image Super-Resolution in the Wild](https://mlsanity.com/paper/2005.03341/) (2020)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition](https://mlsanity.com/paper/2002.01276/) (2020)
- [TextSR: Content-Aware Text Super-Resolution Guided by Recognition](https://mlsanity.com/paper/1909.07113/) (2019)
- [C3-STISR: Scene Text Image Super-resolution with Triple Clues](https://mlsanity.com/paper/2204.14044/) (2022)
- [Edit Probability for Scene Text Recognition](https://mlsanity.com/paper/1805.03384/) (2018)
- [Scene Text Image Super-Resolution via Content Perceptual Loss and Criss-Cross Transformer Blocks](https://mlsanity.com/paper/2210.06924/) (2022)
- [Binary Document Image Super Resolution for Improved Readability and OCR Performance](https://mlsanity.com/paper/1812.02475/) (2018)
- [Ensemble Super-Resolution with A Reference Dataset](https://mlsanity.com/paper/1905.04696/) (2019)
- [Deep learning-based blind image super-resolution with iterative kernel reconstruction and noise estimation](https://mlsanity.com/paper/2404.16564/) (2024)

## Abstract (arXiv)

Scene text recognition (STR) from high-resolution (HR) images has been significantly successful, however text reading on low-resolution (LR) images is still challenging due to insufficient visual information. Therefore, recently many scene text image super-resolution (STISR) models have been proposed to generate super-resolution (SR) images for the LR ones, then STR is done on the SR images, which thus boosts recognition performance. Nevertheless, these methods have two major weaknesses. On the one hand, STISR approaches may generate imperfect or even erroneous SR images, which mislead the subsequent recognition of STR models. On the other hand, as the STISR and STR models are jointly optimized, to pursue high recognition accuracy, the fidelity of SR images may be spoiled. As a result, neither the recognition performance nor the fidelity of STISR models are desirable. Then, can we achieve both high recognition performance and good fidelity? To this end, in this paper we propose a novel method called IMAGE (the abbreviation of Iterative MutuAl GuidancE) to effectively recognize and recover LR scene text images simultaneously. Concretely, IMAGE consists of a specialized STR model for recognition and a tailored STISR model to recover LR images, which are optimized separately. And we develop an iterative mutual guidance mechanism, with which the STR model provides high-level semantic information as clue to the STISR model for better super-resolution, meanwhile the STISR model offers essential low-level pixel clue to the STR model for more accurate recognition. Extensive experiments on two LR datasets demonstrate the superiority of our method over the existing works on both recognition performance and super-resolution fidelity.