# PEPSI++: Fast and Lightweight Network for Image Inpainting

arXiv: https://arxiv.org/abs/1905.09010 · HTML: https://mlsanity.com/paper/1905.09010/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 24 of 24)

- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance](https://mlsanity.com/paper/2412.12974/) (2024)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Exploring The Effect of High-frequency Components in GANs Training](https://mlsanity.com/paper/2103.11093/) (2021)
- [Generative Convolution Layer for Image Generation](https://mlsanity.com/paper/2111.15171/) (2021)
- [PConv: Simple yet Effective Convolutional Layer for Generative Adversarial Network](https://mlsanity.com/paper/2101.10841/) (2021)
- [Image Inpainting Using Wasserstein Generative Adversarial Imputation Network](https://mlsanity.com/paper/2106.15341/) (2021)
- [A Novel Generator with Auxiliary Branch for Improving GAN Performance](https://mlsanity.com/paper/2112.14968/) (2021)
- [Iterative Facial Image Inpainting Based on an Encoder-Generator Architecture](https://mlsanity.com/paper/2101.07036/) (2021)
- [High-dimensional Assisted Generative Model for Color Image Restoration](https://mlsanity.com/paper/2108.06460/) (2021)
- [Image Generation with Self Pixel-wise Normalization](https://mlsanity.com/paper/2201.10725/) (2022)
- [Region-aware Attention for Image Inpainting](https://mlsanity.com/paper/2204.01004/) (2022)
- [Line Drawing Guided Progressive Inpainting of Mural Damage](https://mlsanity.com/paper/2211.06649/) (2022)
- [The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck](https://mlsanity.com/paper/2211.07959/) (2022)
- [E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space](https://mlsanity.com/paper/2403.12197/) (2024)
- [Rethinking Image Skip Connections in StyleGAN2](https://mlsanity.com/paper/2407.05527/) (2024)
- [Differential-Critic GAN: Generating What You Want by a Cue of Preferences](https://mlsanity.com/paper/2107.06700/) (2021)
- [Content-aware Directed Propagation Network with Pixel Adaptive Kernel Attention](https://mlsanity.com/paper/2107.13144/) (2021)
- [Effective Shortcut Technique for GAN](https://mlsanity.com/paper/2201.11351/) (2022)
- [Improving GANs with a Feature Cycling Generator](https://mlsanity.com/paper/2210.09638/) (2022)
- [WSSL: Weighted Self-supervised Learning Framework For Image-inpainting](https://mlsanity.com/paper/2211.13856/) (2022)
- [AURA : Automatic Mask Generator using Randomized Input Sampling for Object Removal](https://mlsanity.com/paper/2305.07857/) (2023)
- [Face Mask Removal with Region-attentive Face Inpainting](https://mlsanity.com/paper/2409.06845/) (2024)
- [Task-Decoupled Image Inpainting Framework for Class-specific Object Remover](https://mlsanity.com/paper/2410.02894/) (2024)

## Abstract (arXiv)

Among the various generative adversarial network (GAN)-based image inpainting methods, a coarse-to-fine network with a contextual attention module (CAM) has shown remarkable performance. However, owing to two stacked generative networks, the coarse-to-fine network needs numerous computational resources such as convolution operations and network parameters, which result in low speed. To address this problem, we propose a novel network architecture called PEPSI: parallel extended-decoder path for semantic inpainting network, which aims at reducing the hardware costs and improving the inpainting performance. PEPSI consists of a single shared encoding network and parallel decoding networks called coarse and inpainting paths. The coarse path produces a preliminary inpainting result to train the encoding network for the prediction of features for the CAM. Simultaneously, the inpainting path generates higher inpainting quality using the refined features reconstructed via the CAM. In addition, we propose Diet-PEPSI that significantly reduces the network parameters while maintaining the performance. In Diet-PEPSI, to capture the global contextual information with low hardware costs, we propose novel rate-adaptive dilated convolutional layers, which employ the common weights but produce dynamic features depending on the given dilation rates. Extensive experiments comparing the performance with state-of-the-art image inpainting methods demonstrate that both PEPSI and Diet-PEPSI improve the qualitative scores, i.e. the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), as well as significantly reduce hardware costs such as computational time and the number of network parameters.