# Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

arXiv: https://arxiv.org/abs/2109.03426 · HTML: https://mlsanity.com/paper/2109.03426/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 76 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 30 of 59)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition](https://mlsanity.com/paper/1710.10400/) (2017)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [TextBoxes++: A Single-Shot Oriented Scene Text Detector](https://mlsanity.com/paper/1801.02765/) (2018)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes](https://mlsanity.com/paper/1807.01544/) (2018)
- [Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network](https://mlsanity.com/paper/1908.05900/) (2019)
- [Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting](https://mlsanity.com/paper/2007.09482/) (2020)
- [ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)](https://mlsanity.com/paper/1708.09585/) (2017)
- [Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learning](https://mlsanity.com/paper/2006.11476/) (2020)
- [FreeAnchor: Learning to Match Anchors for Visual Object Detection](https://mlsanity.com/paper/1909.02466/) (2019)
- [Rotation-Sensitive Regression for Oriented Scene Text Detection](https://mlsanity.com/paper/1803.05265/) (2018)
- [Detecting Text in Natural Image with Connectionist Text Proposal Network](https://mlsanity.com/paper/1609.03605/) (2016)
- [Video Cloze Procedure for Self-Supervised Spatio-Temporal Learning](https://mlsanity.com/paper/2001.00294/) (2020)
- [Detecting Oriented Text in Natural Images by Linking Segments](https://mlsanity.com/paper/1703.06520/) (2017)
- [Deep Relational Reasoning Graph Network for Arbitrary Shape Text Detection](https://mlsanity.com/paper/2003.07493/) (2020)
- [ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection](https://mlsanity.com/paper/2004.04940/) (2020)
- [Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes](https://mlsanity.com/paper/1904.06535/) (2019)
- [PixelLink: Detecting Scene Text via Instance Segmentation](https://mlsanity.com/paper/1801.01315/) (2018)

## Built on by (most cited first, 23 of 23)

- [Improving Diffusion Models for Scene Text Editing with Dual Encoders](https://mlsanity.com/paper/2304.05568/) (2023)
- [Exploring Stroke-Level Modifications for Scene Text Editing](https://mlsanity.com/paper/2212.01982/) (2022)
- [GLASS: Global to Local Attention for Scene-Text Spotting](https://mlsanity.com/paper/2208.03364/) (2022)
- [TPSNet: Reverse Thinking of Thin Plate Splines for Arbitrary Shape Scene Text Representation](https://mlsanity.com/paper/2110.12826/) (2021)
- [First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending](https://mlsanity.com/paper/2410.10168/) (2024)
- [Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning](https://mlsanity.com/paper/2308.07202/) (2023)
- [Bridging Synthetic and Real Worlds for Pre-training Scene Text Detectors](https://mlsanity.com/paper/2312.05286/) (2023)
- [DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spotting](https://mlsanity.com/paper/2305.19957/) (2023)
- [FASTER: A Font-Agnostic Scene Text Editing and Rendering Framework](https://mlsanity.com/paper/2308.02905/) (2023)
- [Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval](https://mlsanity.com/paper/2408.00441/) (2024)
- [Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2503.18746/) (2025)
- [TextBlockV2: Towards Precise-Detection-Free Scene Text Spotting with Pre-trained Language Model](https://mlsanity.com/paper/2403.10047/) (2024)
- [Towards Natural Language-Based Document Image Retrieval: New Dataset and Benchmark](https://mlsanity.com/paper/2512.20174/) (2025)
- [ChartEye: A Deep Learning Framework for Chart Information Extraction](https://mlsanity.com/paper/2408.16123/) (2024)
- [PBFormer: Capturing Complex Scene Text Shape with Polynomial Band Transformer](https://mlsanity.com/paper/2308.15004/) (2023)
- [A New Multifractal-based Deep Learning Model for Text Mining](https://mlsanity.com/paper/2111.13861/) (2021)
- [UNITS: Unsupervised Intermediate Training Stage for Scene Text Detection](https://mlsanity.com/paper/2205.04683/) (2022)
- [E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flow](https://mlsanity.com/paper/2207.06893/) (2022)
- [A new hazard event classification model via deep learning and multifractal](https://mlsanity.com/paper/2209.05263/) (2022)
- [Self-supervised Scene Text Segmentation with Object-centric Layered Representations Augmented by Text Regions](https://mlsanity.com/paper/2308.13178/) (2023)
- [Towards Training-Free Scene Text Editing](https://mlsanity.com/paper/2603.24571/) (2026)
- [Aggregated Text Transformer for Scene Text Detection](https://mlsanity.com/paper/2211.13984/) (2022)
- [LOGO: Video Text Spotting with Language Collaboration and Glyph Perception Model](https://mlsanity.com/paper/2405.19194/) (2024)

## Abstract (arXiv)

Due to the large success in object detection and instance segmentation, Mask R-CNN attracts great attention and is widely adopted as a strong baseline for arbitrary-shaped scene text detection and spotting. However, two issues remain to be settled. The first is dense text case, which is easy to be neglected but quite practical. There may exist multiple instances in one proposal, which makes it difficult for the mask head to distinguish different instances and degrades the performance. In this work, we argue that the performance degradation results from the learning confusion issue in the mask head. We propose to use an MLP decoder instead of the "deconv-conv" decoder in the mask head, which alleviates the issue and promotes robustness significantly. And we propose instance-aware mask learning in which the mask head learns to predict the shape of the whole instance rather than classify each pixel to text or non-text. With instance-aware mask learning, the mask branch can learn separated and compact masks. The second is that due to large variations in scale and aspect ratio, RPN needs complicated anchor settings, making it hard to maintain and transfer across different datasets. To settle this issue, we propose an adaptive label assignment in which all instances especially those with extreme aspect ratios are guaranteed to be associated with enough anchors. Equipped with these components, the proposed method named MAYOR achieves state-of-the-art performance on five benchmarks including DAST1500, MSRA-TD500, ICDAR2015, CTW1500, and Total-Text.