# Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency

arXiv: https://arxiv.org/abs/1805.11145 · HTML: https://mlsanity.com/paper/1805.11145/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 30 of 41)

- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network](https://mlsanity.com/paper/2105.09188/) (2021)
- [Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment](https://mlsanity.com/paper/2203.04121/) (2022)
- [Unbalanced Feature Transport for Exemplar-based Image Translation](https://mlsanity.com/paper/2106.10482/) (2021)
- [CoMoGAN: continuous model-guided image-to-image translation](https://mlsanity.com/paper/2103.06879/) (2021)
- [AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation](https://mlsanity.com/paper/2102.12593/) (2021)
- [Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2110.04111/) (2021)
- [DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation](https://mlsanity.com/paper/2207.06124/) (2022)
- [On the Road to Online Adaptation for Semantic Image Segmentation](https://mlsanity.com/paper/2203.16195/) (2022)
- [MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation](https://mlsanity.com/paper/2209.11047/) (2022)
- [Bi-level Feature Alignment for Versatile Image Translation and Manipulation](https://mlsanity.com/paper/2107.03021/) (2021)
- [Marginal Contrastive Correspondence for Guided Image Generation](https://mlsanity.com/paper/2204.00442/) (2022)
- [Unaligned Image-to-Image Translation by Learning to Reweight](https://mlsanity.com/paper/2109.11736/) (2021)
- [Few-shot Image Generation via Masked Discrimination](https://mlsanity.com/paper/2210.15194/) (2022)
- [RewriteNet: Reliable Scene Text Editing with Implicit Decomposition of Text Contents and Styles](https://mlsanity.com/paper/2107.11041/) (2021)
- [LAN: Learning to Adapt Noise for Image Denoising](https://mlsanity.com/paper/2412.10651/) (2024)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Auto-regressive Image Synthesis with Integrated Quantization](https://mlsanity.com/paper/2207.10776/) (2022)
- [Defect Transfer GAN: Diverse Defect Synthesis for Data Augmentation](https://mlsanity.com/paper/2302.08366/) (2023)
- [Masked and Adaptive Transformer for Exemplar Based Image Translation](https://mlsanity.com/paper/2303.17123/) (2023)
- [Progressive and Aligned Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/2103.11622/) (2021)
- [StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis](https://mlsanity.com/paper/2104.07098/) (2021)
- [ManiFest: Manifold Deformation for Few-shot Image Translation](https://mlsanity.com/paper/2111.13681/) (2021)
- [MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs](https://mlsanity.com/paper/2303.02906/) (2023)
- [Semi-supervised reference-based sketch extraction using a contrastive learning framework](https://mlsanity.com/paper/2407.14026/) (2024)
- [APRNet: Attention-based Pixel-wise Rendering Network for Photo-Realistic Text Image Generation](https://mlsanity.com/paper/2203.07705/) (2022)
- [StyleFlow For Content-Fixed Image to Image Translation](https://mlsanity.com/paper/2207.01909/) (2022)
- [Multiple GAN Inversion for Exemplar-based Image-to-Image Translation](https://mlsanity.com/paper/2103.14471/) (2021)

## Abstract (arXiv)

Image-to-image translation has recently received significant attention due to advances in deep learning. Most works focus on learning either a one-to-one mapping in an unsupervised way or a many-to-many mapping in a supervised way. However, a more practical setting is many-to-many mapping in an unsupervised way, which is harder due to the lack of supervision and the complex inner- and cross-domain variations. To alleviate these issues, we propose the Exemplar Guided & Semantically Consistent Image-to-image Translation (EGSC-IT) network which conditions the translation process on an exemplar image in the target domain. We assume that an image comprises of a content component which is shared across domains, and a style component specific to each domain. Under the guidance of an exemplar from the target domain we apply Adaptive Instance Normalization to the shared content component, which allows us to transfer the style information of the target domain to the source domain. To avoid semantic inconsistencies during translation that naturally appear due to the large inner- and cross-domain variations, we introduce the concept of feature masks that provide coarse semantic guidance without requiring the use of any semantic labels. Experimental results on various datasets show that EGSC-IT does not only translate the source image to diverse instances in the target domain, but also preserves the semantic consistency during the process.