# TPFNet: A Novel Text In-painting Transformer for Text Removal

arXiv: https://arxiv.org/abs/2210.14461 · HTML: https://mlsanity.com/paper/2210.14461/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [MTRNet: A Generic Scene Text Eraser](https://mlsanity.com/paper/1903.04092/) (2019)
- [EnsNet: Ensconce Text in the Wild](https://mlsanity.com/paper/1812.00723/) (2018)
- [Scene Text Eraser](https://mlsanity.com/paper/1705.02772/) (2017)
- [MTRNet++: One-stage Mask-based Scene Text Eraser](https://mlsanity.com/paper/1912.07183/) (2019)
- [Scene text removal via cascaded text stroke detection and erasing](https://mlsanity.com/paper/2011.09768/) (2020)
- [Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context](https://mlsanity.com/paper/2207.10273/) (2022)

## Built on by (most cited first, 2 of 2)

- [DeepEraser: Deep Iterative Context Mining for Generic Text Eraser](https://mlsanity.com/paper/2402.19108/) (2024)
- [Ethical Framework for Responsible Foundational Models in Medical Imaging](https://mlsanity.com/paper/2406.11868/) (2024)

## Abstract (arXiv)

Text erasure from an image is helpful for various tasks such as image editing and privacy preservation. In this paper, we present TPFNet, a novel one-stage (end-toend) network for text removal from images. Our network has two parts: feature synthesis and image generation. Since noise can be more effectively removed from low-resolution images, part 1 operates on low-resolution images. The output of part 1 is a low-resolution text-free image. Part 2 uses the features learned in part 1 to predict a high-resolution text-free image. In part 1, we use "pyramidal vision transformer" (PVT) as the encoder. Further, we use a novel multi-headed decoder that generates a high-pass filtered image and a segmentation map, in addition to a text-free image. The segmentation branch helps locate the text precisely, and the high-pass branch helps in learning the image structure. To precisely locate the text, TPFNet employs an adversarial loss that is conditional on the segmentation map rather than the input image. On Oxford, SCUT, and SCUT-EnsText datasets, our network outperforms recently proposed networks on nearly all the metrics. For example, on SCUT-EnsText dataset, TPFNet has a PSNR (higher is better) of 39.0 and text-detection precision (lower is better) of 21.1, compared to the best previous technique, which has a PSNR of 32.3 and precision of 53.2. The source code can be obtained from https://github.com/CandleLabAI/TPFNet