# EAFormer: Scene Text Segmentation with Edge-Aware Transformers

arXiv: https://arxiv.org/abs/2407.17020 · HTML: https://mlsanity.com/paper/2407.17020/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 55 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition](https://mlsanity.com/paper/1710.10400/) (2017)
- [Context Prior for Scene Segmentation](https://mlsanity.com/paper/2004.01547/) (2020)
- [Detecting Oriented Text in Natural Images by Linking Segments](https://mlsanity.com/paper/1703.06520/) (2017)
- [Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach](https://mlsanity.com/paper/2011.14021/) (2020)
- [Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation](https://mlsanity.com/paper/1802.08948/) (2018)
- [Deep Direct Regression for Multi-Oriented Scene Text Detection](https://mlsanity.com/paper/1703.08289/) (2017)
- [Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation](https://mlsanity.com/paper/1911.09026/) (2019)
- [COCO_TS Dataset: Pixel-level Annotations Based on Weak Supervision for Scene Text Segmentation](https://mlsanity.com/paper/1904.00818/) (2019)
- [A Two Stage GAN for High Resolution Retinal Image Generation and Segmentation](https://mlsanity.com/paper/1907.12296/) (2019)
- [FETNet: Feature Erasing and Transferring Network for Scene Text Removal](https://mlsanity.com/paper/2306.09593/) (2023)
- [Boundary Guided Semantic Learning for Real-time COVID-19 Lung Infection Segmentation System](https://mlsanity.com/paper/2209.02934/) (2022)
- [Fused Text Segmentation Networks for Multi-oriented Scene Text Detection](https://mlsanity.com/paper/1709.03272/) (2017)
- [Boundary Guided Context Aggregation for Semantic Segmentation](https://mlsanity.com/paper/2110.14587/) (2021)
- [Weakly-Supervised Text Instance Segmentation](https://mlsanity.com/paper/2303.10848/) (2023)
- [Learning Document Image Binarization from Data](https://mlsanity.com/paper/1505.00529/) (2015)

## Built on by (most cited first, 4 of 4)

- [Vision as Unified Multimodal Generation](https://mlsanity.com/paper/2607.06560/) (2026)
- [MonkeyOCRv2: A Visual-Text Foundation Model for Document AI](https://mlsanity.com/paper/2607.11562/) (2026)
- [ConText: Driving In-context Learning for Text Removal and Segmentation](https://mlsanity.com/paper/2506.03799/) (2025)
- [Learning Attribute-aware Representations for Few-shot Scene Text Segmentation](https://mlsanity.com/paper/2504.11164/) (2025)

## Abstract (arXiv)

Scene text segmentation aims at cropping texts from scene images, which is usually used to help generative models edit or remove texts. The existing text segmentation methods tend to involve various text-related supervisions for better performance. However, most of them ignore the importance of text edges, which are significant for downstream applications. In this paper, we propose Edge-Aware Transformers, termed EAFormer, to segment texts more accurately, especially at the edge of texts. Specifically, we first design a text edge extractor to detect edges and filter out edges of non-text areas. Then, we propose an edge-guided encoder to make the model focus more on text edges. Finally, an MLP-based decoder is employed to predict text masks. We have conducted extensive experiments on commonly-used benchmarks to verify the effectiveness of EAFormer. The experimental results demonstrate that the proposed method can perform better than previous methods, especially on the segmentation of text edges. Considering that the annotations of several benchmarks (e.g., COCO_TS and MLT_S) are not accurate enough to fairly evaluate our methods, we have relabeled these datasets. Through experiments, we observe that our method can achieve a higher performance improvement when more accurate annotations are used for training.