# TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network

arXiv: https://arxiv.org/abs/1812.09900 · HTML: https://mlsanity.com/paper/1812.09900/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2018.

## Built on by (most cited first, 16 of 16)

- [ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting](https://mlsanity.com/paper/2105.03620/) (2021)
- [OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition](https://mlsanity.com/paper/2403.19128/) (2024)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [SPTS v2: Single-Point Scene Text Spotting](https://mlsanity.com/paper/2301.01635/) (2023)
- [PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text](https://mlsanity.com/paper/2105.00405/) (2021)
- [PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network](https://mlsanity.com/paper/2104.05458/) (2021)
- [ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spotting](https://mlsanity.com/paper/2211.10578/) (2022)
- [MOST: A Multi-Oriented Scene Text Detector with Localization Refinement](https://mlsanity.com/paper/2104.01070/) (2021)
- [Inverse-like Antagonistic Scene Text Spotting via Reading-Order Estimation and Dynamic Sampling](https://mlsanity.com/paper/2401.03637/) (2024)
- [DEER: Detection-agnostic End-to-End Recognizer for Scene Text Spotting](https://mlsanity.com/paper/2203.05122/) (2022)
- [OCR-RTPS: An OCR-based real-time positioning system for the valet parking](https://mlsanity.com/paper/2212.04116/) (2022)
- [Dynamic Low-Resolution Distillation for Cost-Efficient End-to-End Text Spotting](https://mlsanity.com/paper/2207.06694/) (2022)
- [TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision](https://mlsanity.com/paper/2306.03377/) (2023)
- [FastTextSpotter: A High-Efficiency Transformer for Multilingual Scene Text Spotting](https://mlsanity.com/paper/2408.14998/) (2024)
- [Single Shot Self-Reliant Scene Text Spotter by Decoupled yet Collaborative Detection and Recognition](https://mlsanity.com/paper/2207.07253/) (2022)
- [Hear the Scene: Audio-Enhanced Text Spotting](https://mlsanity.com/paper/2412.19504/) (2024)

## Abstract (arXiv)

Reading text from images remains challenging due to multi-orientation, perspective distortion and especially the curved nature of irregular text. Most of existing approaches attempt to solve the problem in two or multiple stages, which is considered to be the bottleneck to optimize the overall performance. To address this issue, we propose an end-to-end trainable network architecture, named TextNet, which is able to simultaneously localize and recognize irregular text from images. Specifically, we develop a scale-aware attention mechanism to learn multi-scale image features as a backbone network, sharing fully convolutional features and computation for localization and recognition. In text detection branch, we directly generate text proposals in quadrangles, covering oriented, perspective and curved text regions. To preserve text features for recognition, we introduce a perspective RoI transform layer, which can align quadrangle proposals into small feature maps. Furthermore, in order to extract effective features for recognition, we propose to encode the aligned RoI features by RNN into context information, combining spatial attention mechanism to generate text sequences. This overall pipeline is capable of handling both regular and irregular cases. Finally, text localization and recognition tasks can be jointly trained in an end-to-end fashion with designed multi-task loss. Experiments on standard benchmarks show that the proposed TextNet can achieve state-of-the-art performance, and outperform existing approaches on irregular datasets by a large margin.