# Delta-GAN-Encoder: Encoding Semantic Changes for Explicit Image Editing, using Few Synthetic Samples

arXiv: https://arxiv.org/abs/2111.08419 · HTML: https://mlsanity.com/paper/2111.08419/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 30 of 31)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Only a Matter of Style: Age Transformation Using a Style-Based Regression Model](https://mlsanity.com/paper/2102.02754/) (2021)
- [Delta-encoder: an effective sample synthesis method for few-shot object recognition](https://mlsanity.com/paper/1806.04734/) (2018)
- [Improved StyleGAN Embedding: Where are the Good Latents?](https://mlsanity.com/paper/2012.09036/) (2020)
- [Semi-Supervised Learning with Normalizing Flows](https://mlsanity.com/paper/1912.13025/) (2019)
- [Towards Large-Pose Face Frontalization in the Wild](https://mlsanity.com/paper/1704.06244/) (2017)
- [3D Guided Fine-Grained Face Manipulation](https://mlsanity.com/paper/1902.08900/) (2019)
- [Learning Disentangled Representation for Robust Person Re-identification](https://mlsanity.com/paper/1910.12003/) (2019)
- [Emerging Convolutions for Generative Normalizing Flows](https://mlsanity.com/paper/1901.11137/) (2019)
- [OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization](https://mlsanity.com/paper/1905.10836/) (2019)
- [MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking](https://mlsanity.com/paper/1912.07748/) (2019)
- [Face De-occlusion using 3D Morphable Model and Generative Adversarial Network](https://mlsanity.com/paper/1904.06109/) (2019)

## Abstract (arXiv)

Understating and controlling generative models' latent space is a complex task. In this paper, we propose a novel method for learning to control any desired attribute in a pre-trained GAN's latent space, for the purpose of editing synthesized and real-world data samples accordingly. We perform Sim2Real learning, relying on minimal samples to achieve an unlimited amount of continuous precise edits. We present an Autoencoder-based model that learns to encode the semantics of changes between images as a basis for editing new samples later on, achieving precise desired results - example shown in Fig. 1. While previous editing methods rely on a known structure of latent spaces (e.g., linearity of some semantics in StyleGAN), our method inherently does not require any structural constraints. We demonstrate our method in the domain of facial imagery: editing different expressions, poses, and lighting attributes, achieving state-of-the-art results.