# IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion arXiv: https://arxiv.org/abs/2209.10811 ยท HTML: https://mlsanity.com/paper/2209.10811/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 38 of its 45 references that are in the map. - First posted on arXiv 22 Sep 2022. ## Builds on (most cited first, 30 of 38) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016) - [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019) - [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018) - [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019) - [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019) - [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021) - [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020) - [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019) - [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019) - [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017) - [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018) - [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021) - [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021) - [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020) - [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020) - [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019) - [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016) - [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020) - [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020) - [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020) - [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021) - [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021) - [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019) - [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020) - [Semantic Photo Manipulation with a Generative Image Prior](https://mlsanity.com/paper/2005.07727/) (2020) ## Built on by (most cited first, 8 of 8) - [Generative Unlearning for Any Identity](https://mlsanity.com/paper/2405.09879/) (2024) - [Make It So: Steering StyleGAN for Any Image Inversion and Editing](https://mlsanity.com/paper/2304.14403/) (2023) - [Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization](https://mlsanity.com/paper/2307.00648/) (2023) - [WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement](https://mlsanity.com/paper/2210.09655/) (2022) - [Pedestrian Attribute Editing for Gait Recognition and Anonymization](https://mlsanity.com/paper/2303.05076/) (2023) - [JoIN: Joint GANs Inversion for Intrinsic Image Decomposition](https://mlsanity.com/paper/2305.11321/) (2023) - [Gradual Residuals Alignment: A Dual-Stream Framework for GAN Inversion and Image Attribute Editing](https://mlsanity.com/paper/2402.14398/) (2024) - [OSVE: One Step Video Editing with One Step Diffusion Models](https://mlsanity.com/paper/2607.19895/) (2026) ## Abstract (arXiv) Recently, manipulation of real-world images has been highly elaborated along with the development of Generative Adversarial Networks (GANs) and corresponding encoders, which embed real-world images into the latent space. However, designing encoders of GAN still remains a challenging task due to the trade-off between distortion and perception. In this paper, we point out that the existing encoders try to lower the distortion not only on the interest region, e.g., human facial region but also on the uninterest region, e.g., background patterns and obstacles. However, most uninterest regions in real-world images are located at out-of-distribution (OOD), which are infeasible to be ideally reconstructed by generative models. Moreover, we empirically find that the uninterest region overlapped with the interest region can mangle the original feature of the interest region, e.g., a microphone overlapped with a facial region is inverted into the white beard. As a result, lowering the distortion of the whole image while maintaining the perceptual quality is very challenging. To overcome this trade-off, we propose a simple yet effective encoder training scheme, coined IntereStyle, which facilitates encoding by focusing on the interest region. IntereStyle steers the encoder to disentangle the encodings of the interest and uninterest regions. To this end, we filter the information of the uninterest region iteratively to regulate the negative impact of the uninterest region. We demonstrate that IntereStyle achieves both lower distortion and higher perceptual quality compared to the existing state-of-the-art encoders. Especially, our model robustly conserves features of the original images, which shows the robust image editing and style mixing results. We will release our code with the pre-trained model after the review.