# Optimal Boxes: Boosting End-to-End Scene Text Recognition by Adjusting Annotated Bounding Boxes via Reinforcement Learning

arXiv: https://arxiv.org/abs/2207.11934 · HTML: https://mlsanity.com/paper/2207.11934/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition](https://mlsanity.com/paper/1710.10400/) (2017)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard](https://mlsanity.com/paper/1912.09641/) (2019)
- [Reading Text in the Wild with Convolutional Neural Networks](https://mlsanity.com/paper/1412.1842/) (2014)
- [Robust Scene Text Recognition with Automatic Rectification](https://mlsanity.com/paper/1603.03915/) (2016)
- [TextBoxes: A Fast Text Detector with a Single Deep Neural Network](https://mlsanity.com/paper/1611.06779/) (2016)
- [Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting](https://mlsanity.com/paper/2007.09482/) (2020)
- [FOTS: Fast Oriented Text Spotting with a Unified Network](https://mlsanity.com/paper/1801.01671/) (2018)
- [Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection](https://mlsanity.com/paper/2203.15221/) (2022)
- [Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1707.03985/) (2017)
- [Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting](https://mlsanity.com/paper/2002.06820/) (2020)
- [An end-to-end TextSpotter with Explicit Alignment and Attention](https://mlsanity.com/paper/1803.03474/) (2018)
- [PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network](https://mlsanity.com/paper/2104.05458/) (2021)
- [Active Object Localization with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06015/) (2015)
- [Tree-Structured Reinforcement Learning for Sequential Object Localization](https://mlsanity.com/paper/1703.02710/) (2017)
- [SEE: Towards Semi-Supervised End-to-End Scene Text Recognition](https://mlsanity.com/paper/1712.05404/) (2017)

## Built on by (most cited first, 30 of 33)

- [TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy](https://mlsanity.com/paper/2406.01326/) (2024)
- [TextSquare: Scaling up Text-Centric Visual Instruction Tuning](https://mlsanity.com/paper/2404.12803/) (2024)
- [Harmonizing Visual Text Comprehension and Generation](https://mlsanity.com/paper/2407.16364/) (2024)
- [Effects of Prompt Length on Domain-specific Tasks for Large Language Models](https://mlsanity.com/paper/2502.14255/) (2025)
- [Journey into Automation: Image-Derived Pavement Texture Extraction and Evaluation](https://mlsanity.com/paper/2501.02414/) (2025)
- [A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyond](https://mlsanity.com/paper/2508.11957/) (2025)
- [Regression and Forecasting of U.S. Stock Returns Based on LSTM](https://mlsanity.com/paper/2502.05210/) (2025)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)
- [Efficiently Leveraging Linguistic Priors for Scene Text Spotting](https://mlsanity.com/paper/2402.17134/) (2024)
- [TDRI: Two-Phase Dialogue Refinement and Co-Adaptation for Interactive Image Generation](https://mlsanity.com/paper/2503.17669/) (2025)
- [Hear the Scene: Audio-Enhanced Text Spotting](https://mlsanity.com/paper/2412.19504/) (2024)
- [First-place Solution for Streetscape Shop Sign Recognition Competition](https://mlsanity.com/paper/2501.02811/) (2025)
- [Cross-Modal Synergies: Unveiling the Potential of Motion-Aware Fusion Networks in Handling Dynamic and Static ReID Scenarios](https://mlsanity.com/paper/2502.00665/) (2025)
- [LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data](https://mlsanity.com/paper/2601.09118/) (2026)
- [Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method](https://mlsanity.com/paper/2601.09238/) (2026)
- [MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreading](https://mlsanity.com/paper/2601.20881/) (2026)
- [The Diffusion Duet: Harmonizing Dual Channels with Wavelet Suppression for Image Separation](https://mlsanity.com/paper/2602.13361/) (2026)
- [Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment](https://mlsanity.com/paper/2602.15903/) (2026)
- [Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networks](https://mlsanity.com/paper/2603.00118/) (2026)
- [A Deformable Attention-Based Detection Transformer with Cross-Scale Feature Fusion for Industrial Coil Spring Inspection](https://mlsanity.com/paper/2603.13429/) (2026)
- [Caption Generation for Dongba Paintings via Prompt Learning and Semantic Fusion](https://mlsanity.com/paper/2603.22946/) (2026)
- [Transmittance-Guided Structure-Texture Decomposition for Nighttime Image Dehazing](https://mlsanity.com/paper/2603.29507/) (2026)
- [Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction](https://mlsanity.com/paper/2604.03339/) (2026)
- [Adaptive Slicing-Assisted Hyper Inference for Enhanced Small Object Detection in High-Resolution Imagery](https://mlsanity.com/paper/2604.19233/) (2026)
- [Feature Perturbation Pool-based Fusion Network for Unified Multi-Class Industrial Defect Detection](https://mlsanity.com/paper/2604.19259/) (2026)
- [Lightweight Real-Time Rendering Parameter Optimization via XGBoost-Driven Lookup Tables](https://mlsanity.com/paper/2604.25178/) (2026)
- [Image Classification via Random Dilated Convolution with Multi-Branch Feature Extraction and Context Excitation](https://mlsanity.com/paper/2604.25188/) (2026)
- [Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusion](https://mlsanity.com/paper/2604.27353/) (2026)
- [Local Spatiotemporal Convolutional Network for Robust Gait Recognition](https://mlsanity.com/paper/2605.14548/) (2026)
- [Do You Need Text Rectification? Soft Attention Mask Embedding for Rectification-Free Scene Text Spotting](https://mlsanity.com/paper/2605.18173/) (2026)

## Abstract (arXiv)

Text detection and recognition are essential components of a modern OCR system. Most OCR approaches attempt to obtain accurate bounding boxes of text at the detection stage, which is used as the input of the text recognition stage. We observe that when using tight text bounding boxes as input, a text recognizer frequently fails to achieve optimal performance due to the inconsistency between bounding boxes and deep representations of text recognition. In this paper, we propose Box Adjuster, a reinforcement learning-based method for adjusting the shape of each text bounding box to make it more compatible with text recognition models. Additionally, when dealing with cross-domain problems such as synthetic-to-real, the proposed method significantly reduces mismatches in domain distribution between the source and target domains. Experiments demonstrate that the performance of end-to-end text recognition systems can be improved when using the adjusted bounding boxes as the ground truths for training. Specifically, on several benchmark datasets for scene text understanding, the proposed method outperforms state-of-the-art text spotters by an average of 2.0% F-Score on end-to-end text recognition tasks and 4.6% F-Score on domain adaptation tasks.