# MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking

arXiv: https://arxiv.org/abs/1912.07748 · HTML: https://mlsanity.com/paper/1912.07748/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 16 of 16)

- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization](https://mlsanity.com/paper/2112.01573/) (2021)
- [Anycost GANs for Interactive Image Synthesis and Editing](https://mlsanity.com/paper/2103.03243/) (2021)
- [Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing](https://mlsanity.com/paper/2206.08357/) (2022)
- [Regularising Inverse Problems with Generative Machine Learning Models](https://mlsanity.com/paper/2107.11191/) (2021)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2110.12076/) (2021)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Comparing the latent space of generative models](https://mlsanity.com/paper/2207.06812/) (2022)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [Image Embedding for Denoising Generative Models](https://mlsanity.com/paper/2301.07485/) (2022)
- [Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey](https://mlsanity.com/paper/2303.06302/) (2023)
- [Suppressing simulation bias using multi-modal data](https://mlsanity.com/paper/2104.09684/) (2021)
- [A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration](https://mlsanity.com/paper/2207.10309/) (2022)
- [Delta-GAN-Encoder: Encoding Semantic Changes for Explicit Image Editing, using Few Synthetic Samples](https://mlsanity.com/paper/2111.08419/) (2021)

## Abstract (arXiv)

In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking -- a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications -- anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.