# JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting

arXiv: https://arxiv.org/abs/2107.04281 · HTML: https://mlsanity.com/paper/2107.04281/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 44 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Semantic Image Inpainting with Deep Generative Models](https://mlsanity.com/paper/1607.07539/) (2016)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE](https://mlsanity.com/paper/2103.10022/) (2021)
- [Image inpainting: A review](https://mlsanity.com/paper/1909.06399/) (2019)
- [Auto-Exposure Fusion for Single-Image Shadow Removal](https://mlsanity.com/paper/2103.01255/) (2021)
- [EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining](https://mlsanity.com/paper/2009.09238/) (2020)
- [Learning to Synthesize Motion Blur](https://mlsanity.com/paper/1811.11745/) (2018)
- [Depth Image Inpainting: Improving Low Rank Matrix Completion with Low Gradient Regularization](https://mlsanity.com/paper/1604.05817/) (2016)
- [RankGAN: A Maximum Margin Ranking GAN for Generating Faces](https://mlsanity.com/paper/1812.08196/) (2018)

## Built on by (most cited first, 17 of 17)

- [MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting](https://mlsanity.com/paper/2203.06304/) (2022)
- [Leveraging Inpainting for Single-Image Shadow Removal](https://mlsanity.com/paper/2302.05361/) (2023)
- [Bidirectional Autoregressive Diffusion Model for Dance Generation](https://mlsanity.com/paper/2402.04356/) (2024)
- [HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention](https://mlsanity.com/paper/2402.14185/) (2024)
- [Coarse-to-fine Task-driven Inpainting for Geoscience Images](https://mlsanity.com/paper/2211.11059/) (2022)
- [Structure-Informed Shadow Removal Networks](https://mlsanity.com/paper/2301.03182/) (2023)
- [SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting](https://mlsanity.com/paper/2307.14489/) (2023)
- [SAIR: Learning Semantic-aware Implicit Representation](https://mlsanity.com/paper/2310.09285/) (2023)
- [Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal](https://mlsanity.com/paper/2305.10640/) (2023)
- [Line Drawing Guided Progressive Inpainting of Mural Damage](https://mlsanity.com/paper/2211.06649/) (2022)
- [Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation](https://mlsanity.com/paper/2407.21256/) (2024)
- [Image Completion via Dual-path Cooperative Filtering](https://mlsanity.com/paper/2305.00379/) (2023)
- [Multi-stage Progressive Reasoning for Dunhuang Murals Inpainting](https://mlsanity.com/paper/2305.05902/) (2023)
- [CLII: Visual-Text Inpainting via Cross-Modal Predictive Interaction](https://mlsanity.com/paper/2407.16204/) (2024)
- [AIDOVECL: AI-generated Dataset of Outpainted Vehicles for Eye-level Classification and Localization](https://mlsanity.com/paper/2410.24116/) (2024)
- [Time-variant Image Inpainting via Interactive Distribution Transition Estimation](https://mlsanity.com/paper/2506.23461/) (2025)
- [From Physical Degradation Models to Task-Aware All-in-One Image Restoration](https://mlsanity.com/paper/2601.10192/) (2026)

## Abstract (arXiv)

Image inpainting aims to restore the missing regions of corrupted images and make the recovery result identical to the originally complete image, which is different from the common generative task emphasizing the naturalness or realism of generated images. Nevertheless, existing works usually regard it as a pure generation problem and employ cutting-edge deep generative techniques to address it. The generative networks can fill the main missing parts with realistic contents but usually distort the local structures or introduce obvious artifacts. In this paper, for the first time, we formulate image inpainting as a mix of two problems, predictive filtering and deep generation. Predictive filtering is good at preserving local structures and removing artifacts but falls short to complete the large missing regions. The deep generative network can fill the numerous missing pixels based on the understanding of the whole scene but hardly restores the details identical to the original ones. To make use of their respective advantages, we propose the joint predictive filtering and generative network (JPGNet) that contains three branches: predictive filtering & uncertainty network (PFUNet), deep generative network, and uncertainty-aware fusion network (UAFNet). The PFUNet can adaptively predict pixel-wise kernels for filtering-based inpainting according to the input image and output an uncertainty map. This map indicates the pixels should be processed by filtering or generative networks, which is further fed to the UAFNet for a smart combination between filtering and generative results. Note that, our method as a novel inpainting framework can benefit any existing generation-based methods. We validate our method on three public datasets, Dunhuang, Places2, and CelebA, and demonstrate that our method can enhance three state-of-the-art generative methods significantly with slightly extra time costs.