# Diverse Inpainting and Editing with GAN Inversion

arXiv: https://arxiv.org/abs/2307.15033 · HTML: https://mlsanity.com/paper/2307.15033/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 44 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 30 of 35)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [High-Fidelity GAN Inversion for Image Attribute Editing](https://mlsanity.com/paper/2109.06590/) (2021)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Inst-Inpaint: Instructing to Remove Objects with Diffusion Models](https://mlsanity.com/paper/2304.03246/) (2023)
- [Image-to-image Translation via Hierarchical Style Disentanglement](https://mlsanity.com/paper/2103.01456/) (2021)
- [ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes](https://mlsanity.com/paper/1803.10562/) (2018)
- [High-Fidelity and Arbitrary Face Editing](https://mlsanity.com/paper/2103.15814/) (2021)

## Built on by (most cited first, 20 of 20)

- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting](https://mlsanity.com/paper/2503.08677/) (2025)
- [CLIPAway: Harmonizing Focused Embeddings for Removing Objects via Diffusion Models](https://mlsanity.com/paper/2406.09368/) (2024)
- [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024)
- [NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing](https://mlsanity.com/paper/2406.06523/) (2024)
- [Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation](https://mlsanity.com/paper/2504.05672/) (2025)
- [Warping the Residuals for Image Editing with StyleGAN](https://mlsanity.com/paper/2312.11422/) (2023)
- [Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting](https://mlsanity.com/paper/2403.18186/) (2024)
- [Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images](https://mlsanity.com/paper/2409.20530/) (2024)
- [Zero-shot Text-guided Infinite Image Synthesis with LLM guidance](https://mlsanity.com/paper/2407.12642/) (2024)
- [Learning Position-Aware Implicit Neural Network for Real-World Face Inpainting](https://mlsanity.com/paper/2401.10537/) (2024)
- [Adaptive Segmentation-Based Initialization for Steered Mixture of Experts Image Regression](https://mlsanity.com/paper/2409.10101/) (2024)
- [Identity Preserving 3D Head Stylization with Multiview Score Distillation](https://mlsanity.com/paper/2411.13536/) (2024)
- [Diverse Semantic Image Editing with Style Codes](https://mlsanity.com/paper/2309.13975/) (2023)
- [DETER: Detecting Edited Regions for Deterring Generative Manipulations](https://mlsanity.com/paper/2312.10539/) (2023)
- [Rethinking Image Skip Connections in StyleGAN2](https://mlsanity.com/paper/2407.05527/) (2024)
- [FaithFill: Faithful Inpainting for Object Completion Using a Single Reference Image](https://mlsanity.com/paper/2406.07865/) (2024)
- [MGAN-CRCM: A Novel Multiple Generative Adversarial Network and Coarse-Refinement Based Cognizant Method for Image Inpainting](https://mlsanity.com/paper/2412.19000/) (2024)
- [KAO: Kernel-Adaptive Optimization in Diffusion for Satellite Image](https://mlsanity.com/paper/2511.02462/) (2025)
- [Reversible Inversion for Training-Free Exemplar-guided Image Editing](https://mlsanity.com/paper/2512.01382/) (2025)

## Abstract (arXiv)

Recent inversion methods have shown that real images can be inverted into StyleGAN's latent space and numerous edits can be achieved on those images thanks to the semantically rich feature representations of well-trained GAN models. However, extensive research has also shown that image inversion is challenging due to the trade-off between high-fidelity reconstruction and editability. In this paper, we tackle an even more difficult task, inverting erased images into GAN's latent space for realistic inpaintings and editings. Furthermore, by augmenting inverted latent codes with different latent samples, we achieve diverse inpaintings. Specifically, we propose to learn an encoder and mixing network to combine encoded features from erased images with StyleGAN's mapped features from random samples. To encourage the mixing network to utilize both inputs, we train the networks with generated data via a novel set-up. We also utilize higher-rate features to prevent color inconsistencies between the inpainted and unerased parts. We run extensive experiments and compare our method with state-of-the-art inversion and inpainting methods. Qualitative metrics and visual comparisons show significant improvements.