# High-Fidelity Image Inpainting with GAN Inversion

arXiv: https://arxiv.org/abs/2208.11850 · HTML: https://mlsanity.com/paper/2208.11850/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 51 references that are in the map.
- First posted on arXiv 25 Aug 2022.

## Builds on (most cited first, 30 of 44)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020)
- [Coherent Semantic Attention for Image Inpainting](https://mlsanity.com/paper/1905.12384/) (2019)

## Built on by (most cited first, 15 of 15)

- [Inst-Inpaint: Instructing to Remove Objects with Diffusion Models](https://mlsanity.com/paper/2304.03246/) (2023)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [PromptFix: You Prompt and We Fix the Photo](https://mlsanity.com/paper/2405.16785/) (2024)
- [OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting](https://mlsanity.com/paper/2503.08677/) (2025)
- [Blind Image Restoration via Fast Diffusion Inversion](https://mlsanity.com/paper/2405.19572/) (2024)
- [Diverse Inpainting and Editing with GAN Inversion](https://mlsanity.com/paper/2307.15033/) (2023)
- [MaGIC: Multi-modality Guided Image Completion](https://mlsanity.com/paper/2305.11818/) (2023)
- [Flow-Guided Diffusion for Video Inpainting](https://mlsanity.com/paper/2311.15368/) (2023)
- [Deficiency-Aware Masked Transformer for Video Inpainting](https://mlsanity.com/paper/2307.08629/) (2023)
- [Neural Scene Designer: Self-Styled Semantic Image Manipulation](https://mlsanity.com/paper/2509.01405/) (2025)
- [Diverse Semantic Image Editing with Style Codes](https://mlsanity.com/paper/2309.13975/) (2023)
- [Task-Decoupled Image Inpainting Framework for Class-specific Object Remover](https://mlsanity.com/paper/2410.02894/) (2024)
- [High-Fidelity Image Inpainting with Multimodal Guided GAN Inversion](https://mlsanity.com/paper/2504.12844/) (2025)
- [Online Estimation of Table-Top Grown Strawberry Mass in Field Conditions with Occlusions](https://mlsanity.com/paper/2507.23487/) (2025)
- [CoIn: Comprehensive 2D-3D Inpainting with Gaussian Splatting Guidance](https://mlsanity.com/paper/2606.27584/) (2026)

## Abstract (arXiv)

Image inpainting seeks a semantically consistent way to recover the corrupted image in the light of its unmasked content. Previous approaches usually reuse the well-trained GAN as effective prior to generate realistic patches for missing holes with GAN inversion. Nevertheless, the ignorance of a hard constraint in these algorithms may yield the gap between GAN inversion and image inpainting. Addressing this problem, in this paper, we devise a novel GAN inversion model for image inpainting, dubbed InvertFill, mainly consisting of an encoder with a pre-modulation module and a GAN generator with F&W+ latent space. Within the encoder, the pre-modulation network leverages multi-scale structures to encode more discriminative semantics into style vectors. In order to bridge the gap between GAN inversion and image inpainting, F&W+ latent space is proposed to eliminate glaring color discrepancy and semantic inconsistency. To reconstruct faithful and photorealistic images, a simple yet effective Soft-update Mean Latent module is designed to capture more diverse in-domain patterns that synthesize high-fidelity textures for large corruptions. Comprehensive experiments on four challenging datasets, including Places2, CelebA-HQ, MetFaces, and Scenery, demonstrate that our InvertFill outperforms the advanced approaches qualitatively and quantitatively and supports the completion of out-of-domain images well.