# Improving Scene Text Image Super-resolution via Dual Prior Modulation Network

arXiv: https://arxiv.org/abs/2302.10414 · HTML: https://mlsanity.com/paper/2302.10414/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Conformer: Local Features Coupling Global Representations for Visual Recognition](https://mlsanity.com/paper/2105.03889/) (2021)
- [Scene Text Detection and Recognition: The Deep Learning Era](https://mlsanity.com/paper/1811.04256/) (2018)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [Contextual Transformer Networks for Visual Recognition](https://mlsanity.com/paper/2107.12292/) (2021)
- [Editing Text in the Wild](https://mlsanity.com/paper/1908.03047/) (2019)
- [Scene Text Image Super-Resolution in the Wild](https://mlsanity.com/paper/2005.03341/) (2020)
- [Text Prior Guided Scene Text Image Super-resolution](https://mlsanity.com/paper/2106.15368/) (2021)
- [A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution](https://mlsanity.com/paper/2203.09388/) (2022)
- [Text Gestalt: Stroke-Aware Scene Text Image Super-Resolution](https://mlsanity.com/paper/2112.08171/) (2021)
- [Beyond Fixation: Dynamic Window Visual Transformer](https://mlsanity.com/paper/2203.12856/) (2022)
- [MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning](https://mlsanity.com/paper/2112.06558/) (2021)
- [Scene Text Retrieval via Joint Text Detection and Similarity Learning](https://mlsanity.com/paper/2104.01552/) (2021)
- [Binary Document Image Super Resolution for Improved Readability and OCR Performance](https://mlsanity.com/paper/1812.02475/) (2018)

## Built on by (most cited first, 15 of 15)

- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2308.06743/) (2023)
- [Text Image Inpainting via Global Structure-Guided Diffusion Models](https://mlsanity.com/paper/2401.14832/) (2024)
- [Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing](https://mlsanity.com/paper/2402.03082/) (2024)
- [Recognition-Guided Diffusion Model for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2311.13317/) (2023)
- [PEAN: A Diffusion-Based Prior-Enhanced Attention Network for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2311.17955/) (2023)
- [PriOr-Flow: Enhancing Primitive Panoramic Optical Flow with Orthogonal View](https://mlsanity.com/paper/2506.23897/) (2025)
- [TEXTS-Diff: TEXTS-Aware Diffusion Model for Real-World Text Image Super-Resolution](https://mlsanity.com/paper/2601.17340/) (2026)
- [One-stage Low-resolution Text Recognition with High-resolution Knowledge Transfer](https://mlsanity.com/paper/2308.02770/) (2023)
- [Enhanced Generative Structure Prior for Chinese Text Image Super-resolution](https://mlsanity.com/paper/2508.07537/) (2025)
- [MF-LPR2: Multi-Frame License Plate Image Restoration and Recognition using Optical Flow](https://mlsanity.com/paper/2508.14797/) (2025)
- [NCAP: Scene Text Image Super-Resolution with Non-CAtegorical Prior](https://mlsanity.com/paper/2504.00410/) (2025)
- [TextSR: Diffusion Super-Resolution with Multilingual OCR Guidance](https://mlsanity.com/paper/2505.23119/) (2025)
- [PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution](https://mlsanity.com/paper/2605.13027/) (2026)
- [TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution](https://mlsanity.com/paper/2609.29240/) (2026)

## Abstract (arXiv)

Scene text image super-resolution (STISR) aims to simultaneously increase the resolution and legibility of the text images, and the resulting images will significantly affect the performance of downstream tasks. Although numerous progress has been made, existing approaches raise two crucial issues: (1) They neglect the global structure of the text, which bounds the semantic determinism of the scene text. (2) The priors, e.g., text prior or stroke prior, employed in existing works, are extracted from pre-trained text recognizers. That said, such priors suffer from the domain gap including low resolution and blurriness caused by poor imaging conditions, leading to incorrect guidance. Our work addresses these gaps and proposes a plug-and-play module dubbed Dual Prior Modulation Network (DPMN), which leverages dual image-level priors to bring performance gain over existing approaches. Specifically, two types of prior-guided refinement modules, each using the text mask or graphic recognition result of the low-quality SR image from the preceding layer, are designed to improve the structural clarity and semantic accuracy of the text, respectively. The following attention mechanism hence modulates two quality-enhanced images to attain a superior SR result. Extensive experiments validate that our method improves the image quality and boosts the performance of downstream tasks over five typical approaches on the benchmark. Substantial visualizations and ablation studies demonstrate the advantages of the proposed DPMN. Code is available at: https://github.com/jdfxzzy/DPMN.