# Optimizing Latent Space Directions For GAN-based Local Image Editing

arXiv: https://arxiv.org/abs/2111.12583 · HTML: https://mlsanity.com/paper/2111.12583/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 27 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Rewriting a Deep Generative Model](https://mlsanity.com/paper/2007.15646/) (2020)
- [Controlling generative models with continuous factors of variations](https://mlsanity.com/paper/2001.10238/) (2020)
- [Using latent space regression to analyze and leverage compositionality in GANs](https://mlsanity.com/paper/2103.10426/) (2021)
- [Low-Rank Subspaces in GANs](https://mlsanity.com/paper/2106.04488/) (2021)
- [Linear Semantics in Generative Adversarial Networks](https://mlsanity.com/paper/2104.00487/) (2021)
- [GarmentGAN: Photo-realistic Adversarial Fashion Transfer](https://mlsanity.com/paper/2003.01894/) (2020)
- [Barbershop: GAN-based Image Compositing using Segmentation Masks](https://mlsanity.com/paper/2106.01505/) (2021)
- [Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks](https://mlsanity.com/paper/2105.08222/) (2021)
- [Aesthetics of Neural Network Art](https://mlsanity.com/paper/1903.05696/) (2019)

## Built on by (most cited first, 5 of 5)

- [A Unified Framework for Generative Data Augmentation: A Comprehensive Survey](https://mlsanity.com/paper/2310.00277/) (2023)
- [Class Attribute Inference Attacks: Inferring Sensitive Class Information by Diffusion-Based Attribute Manipulations](https://mlsanity.com/paper/2303.09289/) (2023)
- [Discovering Class-Specific GAN Controls for Semantic Image Synthesis](https://mlsanity.com/paper/2212.01455/) (2022)
- [3D GANs and Latent Space: A comprehensive survey](https://mlsanity.com/paper/2304.03932/) (2023)
- [Editable Stain Transformation Of Histological Images Using Unpaired GANs](https://mlsanity.com/paper/2312.03647/) (2023)

## Abstract (arXiv)

Generative Adversarial Network (GAN) based localized image editing can suffer from ambiguity between semantic attributes. We thus present a novel objective function to evaluate the locality of an image edit. By introducing the supervision from a pre-trained segmentation network and optimizing the objective function, our framework, called Locally Effective Latent Space Direction (LELSD), is applicable to any dataset and GAN architecture. Our method is also computationally fast and exhibits a high extent of disentanglement, which allows users to interactively perform a sequence of edits on an image. Our experiments on both GAN-generated and real images qualitatively demonstrate the high quality and advantages of our method.