# Arbitrary Shape Text Detection via Boundary Transformer

arXiv: https://arxiv.org/abs/2205.05320 · HTML: https://mlsanity.com/paper/2205.05320/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 67 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [TextBoxes: A Fast Text Detector with a Single Deep Neural Network](https://mlsanity.com/paper/1611.06779/) (2016)
- [TextBoxes++: A Single-Shot Oriented Scene Text Detector](https://mlsanity.com/paper/1801.02765/) (2018)
- [TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes](https://mlsanity.com/paper/1807.01544/) (2018)
- [Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network](https://mlsanity.com/paper/1908.05900/) (2019)
- [Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting](https://mlsanity.com/paper/2007.09482/) (2020)
- [Deep Snake for Real-Time Instance Segmentation](https://mlsanity.com/paper/2001.01629/) (2020)
- [FOTS: Fast Oriented Text Spotting with a Unified Network](https://mlsanity.com/paper/1801.01671/) (2018)
- [Fast Interactive Object Annotation with Curve-GCN](https://mlsanity.com/paper/1903.06874/) (2019)
- [Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion](https://mlsanity.com/paper/2202.10304/) (2022)
- [Rotation-Sensitive Regression for Oriented Scene Text Detection](https://mlsanity.com/paper/1803.05265/) (2018)
- [Detecting Oriented Text in Natural Images by Linking Segments](https://mlsanity.com/paper/1703.06520/) (2017)
- [Fourier Contour Embedding for Arbitrary-Shaped Text Detection](https://mlsanity.com/paper/2104.10442/) (2021)
- [Deep Relational Reasoning Graph Network for Arbitrary Shape Text Detection](https://mlsanity.com/paper/2003.07493/) (2020)
- [ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection](https://mlsanity.com/paper/2004.04940/) (2020)
- [Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes](https://mlsanity.com/paper/1904.06535/) (2019)
- [All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting](https://mlsanity.com/paper/1911.09550/) (2019)
- [Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection](https://mlsanity.com/paper/2203.15221/) (2022)
- [Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting](https://mlsanity.com/paper/2002.06820/) (2020)

## Built on by (most cited first, 28 of 28)

- [HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation](https://mlsanity.com/paper/2505.04512/) (2025)
- [DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer](https://mlsanity.com/paper/2207.04491/) (2022)
- [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023)
- [Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation](https://mlsanity.com/paper/2401.17904/) (2024)
- [DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework](https://mlsanity.com/paper/2503.14880/) (2025)
- [OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation](https://mlsanity.com/paper/2506.01801/) (2025)
- [Turning a CLIP Model into a Scene Text Spotter](https://mlsanity.com/paper/2308.10408/) (2023)
- [Inverse-like Antagonistic Scene Text Spotting via Reading-Order Estimation and Dynamic Sampling](https://mlsanity.com/paper/2401.03637/) (2024)
- [Bridging the Gap Between End-to-End and Two-Step Text Spotting](https://mlsanity.com/paper/2404.04624/) (2024)
- [Seeing Text in the Dark: Algorithm and Benchmark](https://mlsanity.com/paper/2404.08965/) (2024)
- [DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spotting](https://mlsanity.com/paper/2305.19957/) (2023)
- [CBNet: A Plug-and-Play Network for Segmentation-Based Scene Text Detection](https://mlsanity.com/paper/2212.02340/) (2022)
- [Text Detection Forgot About Document OCR](https://mlsanity.com/paper/2210.07903/) (2022)
- [MixNet: Toward Accurate Detection of Challenging Scene Text in the Wild](https://mlsanity.com/paper/2308.12817/) (2023)
- [Spotlight Text Detector: Spotlight on Candidate Regions Like a Camera](https://mlsanity.com/paper/2409.16820/) (2024)
- [LRANet++: Low-Rank Approximation Network for Accurate and Efficient Text Spotting](https://mlsanity.com/paper/2511.05818/) (2025)
- [Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding](https://mlsanity.com/paper/2511.23071/) (2025)
- [TextMamba: Scene Text Detector with Mamba](https://mlsanity.com/paper/2512.06657/) (2025)
- [Scene Text Recognition with Single-Point Decoding Network](https://mlsanity.com/paper/2209.01914/) (2022)
- [Artistic-style text detector and a new Movie-Poster dataset](https://mlsanity.com/paper/2406.16307/) (2024)
- [Box2Poly: Memory-Efficient Polygon Prediction of Arbitrarily Shaped and Rotated Text](https://mlsanity.com/paper/2309.11248/) (2023)
- [BPDO:Boundary Points Dynamic Optimization for Arbitrary Shape Scene Text Detection](https://mlsanity.com/paper/2401.09997/) (2024)
- [Task-Specific Adaptation of Segmentation Foundation Model via Prompt Learning](https://mlsanity.com/paper/2403.09199/) (2024)
- [Video-Language Alignment via Spatio-Temporal Graph Transformer](https://mlsanity.com/paper/2407.11677/) (2024)
- [ROSA: Addressing text understanding challenges in photographs via ROtated SAmpling](https://mlsanity.com/paper/2506.03665/) (2025)
- [Inverse Scene Text Removal](https://mlsanity.com/paper/2506.21002/) (2025)
- [Text-Pass Filter: An Efficient Scene Text Detector](https://mlsanity.com/paper/2601.18098/) (2026)
- [ET-SAM: Efficient Point Prompt Prediction in SAM for Unified Scene Text Detection and Layout Analysis](https://mlsanity.com/paper/2603.25168/) (2026)

## Abstract (arXiv)

In arbitrary shape text detection, locating accurate text boundaries is challenging and non-trivial. Existing methods often suffer from indirect text boundary modeling or complex post-processing. In this paper, we systematically present a unified coarse-to-fine framework via boundary learning for arbitrary shape text detection, which can accurately and efficiently locate text boundaries without post-processing. In our method, we explicitly model the text boundary via an innovative iterative boundary transformer in a coarse-to-fine manner. In this way, our method can directly gain accurate text boundaries and abandon complex post-processing to improve efficiency. Specifically, our method mainly consists of a feature extraction backbone, a boundary proposal module, and an iteratively optimized boundary transformer module. The boundary proposal module consisting of multi-layer dilated convolutions will compute important prior information (including classification map, distance field, and direction field) for generating coarse boundary proposals while guiding the boundary transformer's optimization. The boundary transformer module adopts an encoder-decoder structure, in which the encoder is constructed by multi-layer transformer blocks with residual connection while the decoder is a simple multi-layer perceptron network (MLP). Under the guidance of prior information, the boundary transformer module will gradually refine the coarse boundary proposals via iterative boundary deformation. Furthermore, we propose a novel boundary energy loss (BEL) which introduces an energy minimization constraint and an energy monotonically decreasing constraint to further optimize and stabilize the learning of boundary refinement. Extensive experiments on publicly available and challenging datasets demonstrate the state-of-the-art performance and promising efficiency of our method.