# InstaGAN: Instance-aware Image-to-Image Translation

arXiv: https://arxiv.org/abs/1812.10889 · HTML: https://mlsanity.com/paper/1812.10889/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2018.

## Built on by (most cited first, 30 of 37)

- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition](https://mlsanity.com/paper/2103.16019/) (2021)
- [StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis](https://mlsanity.com/paper/2206.09479/) (2022)
- [InstaFormer: Instance-Aware Image-to-Image Translation with Transformer](https://mlsanity.com/paper/2203.16248/) (2022)
- [CoMoGAN: continuous model-guided image-to-image translation](https://mlsanity.com/paper/2103.06879/) (2021)
- [Region-Aware Diffusion for Zero-shot Text-driven Image Editing](https://mlsanity.com/paper/2302.11797/) (2023)
- [Memory-guided Unsupervised Image-to-image Translation](https://mlsanity.com/paper/2104.05170/) (2021)
- [Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks](https://mlsanity.com/paper/2103.05422/) (2021)
- [Learning Hierarchical Image Segmentation For Recognition and By Recognition](https://mlsanity.com/paper/2210.00314/) (2022)
- [ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods](https://mlsanity.com/paper/2110.02871/) (2021)
- [Unaligned Image-to-Image Translation by Learning to Reweight](https://mlsanity.com/paper/2109.11736/) (2021)
- [Metaethical Perspectives on 'Benchmarking' AI Ethics](https://mlsanity.com/paper/2204.05151/) (2022)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Images](https://mlsanity.com/paper/2106.13974/) (2021)
- [ManiFest: Manifold Deformation for Few-shot Image Translation](https://mlsanity.com/paper/2111.13681/) (2021)
- [Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation](https://mlsanity.com/paper/2602.15383/) (2026)
- [Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling](https://mlsanity.com/paper/2212.02090/) (2022)
- [Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing](https://mlsanity.com/paper/2111.12346/) (2021)
- [Panoptic-aware Image-to-Image Translation](https://mlsanity.com/paper/2112.01926/) (2021)
- [SHUNIT: Style Harmonization for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2301.04685/) (2023)
- [Partially fake it till you make it: mixing real and fake thermal images for improved object detection](https://mlsanity.com/paper/2106.13603/) (2021)
- [BioGAN: An unpaired GAN-based image to image translation model for microbiological images](https://mlsanity.com/paper/2306.06217/) (2023)
- [Leveraging Local Domains for Image-to-Image Translation](https://mlsanity.com/paper/2109.04468/) (2021)
- [Culture-to-Culture Image Translation and User Evaluation](https://mlsanity.com/paper/2201.01565/) (2022)
- [DSI2I: Dense Style for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2212.13253/) (2022)
- [Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds](https://mlsanity.com/paper/2302.07661/) (2023)
- [Few-Shot Unsupervised Image-to-Image Translation on complex scenes](https://mlsanity.com/paper/2106.03770/) (2021)
- [Global and Local Alignment Networks for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2111.10346/) (2021)

## Abstract (arXiv)

Unsupervised image-to-image translation has gained considerable attention due to the recent impressive progress based on generative adversarial networks (GANs). However, previous methods often fail in challenging cases, in particular, when an image has multiple target instances and a translation task involves significant changes in shape, e.g., translating pants to skirts in fashion images. To tackle the issues, we propose a novel method, coined instance-aware GAN (InstaGAN), that incorporates the instance information (e.g., object segmentation masks) and improves multi-instance transfiguration. The proposed method translates both an image and the corresponding set of instance attributes while maintaining the permutation invariance property of the instances. To this end, we introduce a context preserving loss that encourages the network to learn the identity function outside of target instances. We also propose a sequential mini-batch inference/training technique that handles multiple instances with a limited GPU memory and enhances the network to generalize better for multiple instances. Our comparative evaluation demonstrates the effectiveness of the proposed method on different image datasets, in particular, in the aforementioned challenging cases. Code and results are available in https://github.com/sangwoomo/instagan