# Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing

arXiv: https://arxiv.org/abs/2206.08357 · HTML: https://mlsanity.com/paper/2206.08357/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 65 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 30 of 59)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)

## Built on by (most cited first, 30 of 39)

- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2302.12228/) (2023)
- [ReNoise: Real Image Inversion Through Iterative Noising](https://mlsanity.com/paper/2403.14602/) (2024)
- [Generalizing Dataset Distillation via Deep Generative Prior](https://mlsanity.com/paper/2305.01649/) (2023)
- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models](https://mlsanity.com/paper/2307.06925/) (2023)
- [Orthogonal Adaptation for Modular Customization of Diffusion Models](https://mlsanity.com/paper/2312.02432/) (2023)
- [TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models](https://mlsanity.com/paper/2408.00735/) (2024)
- [VToonify: Controllable High-Resolution Portrait Video Style Transfer](https://mlsanity.com/paper/2209.11224/) (2022)
- [LoRACLR: Contrastive Adaptation for Customization of Diffusion Models](https://mlsanity.com/paper/2412.09622/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces](https://mlsanity.com/paper/2303.06146/) (2023)
- [Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint](https://mlsanity.com/paper/2211.11448/) (2022)
- [VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs](https://mlsanity.com/paper/2303.15893/) (2023)
- [3D-aware Blending with Generative NeRFs](https://mlsanity.com/paper/2302.06608/) (2023)
- [CLIP-Guided StyleGAN Inversion for Text-Driven Real Image Editing](https://mlsanity.com/paper/2307.08397/) (2023)
- [Cross Initialization for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2312.15905/) (2023)
- [Make It So: Steering StyleGAN for Any Image Inversion and Editing](https://mlsanity.com/paper/2304.14403/) (2023)
- [What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion](https://mlsanity.com/paper/2301.12141/) (2023)
- [Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generation](https://mlsanity.com/paper/2312.08195/) (2023)
- [NaRCan: Natural Refined Canonical Image with Integration of Diffusion Prior for Video Editing](https://mlsanity.com/paper/2406.06523/) (2024)
- [One-shot Neural Face Reenactment via Finding Directions in GAN's Latent Space](https://mlsanity.com/paper/2402.03553/) (2024)
- [GlassesGAN: Eyewear Personalization using Synthetic Appearance Discovery and Targeted Subspace Modeling](https://mlsanity.com/paper/2210.14145/) (2022)
- [CoralStyleCLIP: Co-optimized Region and Layer Selection for Image Editing](https://mlsanity.com/paper/2303.05031/) (2023)
- [Zero-shot racially balanced dataset generation using an existing biased StyleGAN2](https://mlsanity.com/paper/2305.07710/) (2023)
- [LSAP: Rethinking Inversion Fidelity, Perception and Editability in GAN Latent Space](https://mlsanity.com/paper/2209.12746/) (2022)
- [WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement](https://mlsanity.com/paper/2210.09655/) (2022)
- [Out-of-domain GAN inversion via Invertibility Decomposition for Photo-Realistic Human Face Manipulation](https://mlsanity.com/paper/2212.09262/) (2022)
- [StyleCineGAN: Landscape Cinemagraph Generation using a Pre-trained StyleGAN](https://mlsanity.com/paper/2403.14186/) (2024)

## Abstract (arXiv)

Existing GAN inversion and editing methods work well for aligned objects with a clean background, such as portraits and animal faces, but often struggle for more difficult categories with complex scene layouts and object occlusions, such as cars, animals, and outdoor images. We propose a new method to invert and edit such complex images in the latent space of GANs, such as StyleGAN2. Our key idea is to explore inversion with a collection of layers, spatially adapting the inversion process to the difficulty of the image. We learn to predict the "invertibility" of different image segments and project each segment into a latent layer. Easier regions can be inverted into an earlier layer in the generator's latent space, while more challenging regions can be inverted into a later feature space. Experiments show that our method obtains better inversion results compared to the recent approaches on complex categories, while maintaining downstream editability. Please refer to our project page at https://www.cs.cmu.edu/~SAMInversion.