# Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translation

arXiv: https://arxiv.org/abs/2003.00273 · HTML: https://mlsanity.com/paper/2003.00273/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Feb 2020.

## Built on by (most cited first, 30 of 60)

- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Unpaired Image-to-Image Translation via Neural Schr"odinger Bridge](https://mlsanity.com/paper/2305.15086/) (2023)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study](https://mlsanity.com/paper/2311.00265/) (2023)
- [CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwannoma and Cochlea Segmentation](https://mlsanity.com/paper/2201.02831/) (2022)
- [FedMed-GAN: Federated Domain Translation on Unsupervised Cross-Modality Brain Image Synthesis](https://mlsanity.com/paper/2201.08953/) (2022)
- [AesUST: Towards Aesthetic-Enhanced Universal Style Transfer](https://mlsanity.com/paper/2208.13016/) (2022)
- [Unsupervised Domain Adaptation in Semantic Segmentation Based on Pixel Alignment and Self-Training](https://mlsanity.com/paper/2109.14219/) (2021)
- [Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention](https://mlsanity.com/paper/2104.14374/) (2021)
- [Unaligned Image-to-Image Translation by Learning to Reweight](https://mlsanity.com/paper/2109.11736/) (2021)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining](https://mlsanity.com/paper/2403.11340/) (2024)
- [Towards Robust Event-based Networks for Nighttime via Unpaired Day-to-Night Event Translation](https://mlsanity.com/paper/2407.10703/) (2024)
- [Maximum Spatial Perturbation Consistency for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2203.12707/) (2022)
- [C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation](https://mlsanity.com/paper/2411.10788/) (2024)
- [MVAR: Visual Autoregressive Modeling with Scale and Spatial Markovian Conditioning](https://mlsanity.com/paper/2505.12742/) (2025)
- [TransferI2I: Transfer Learning for Image-to-Image Translation from Small Datasets](https://mlsanity.com/paper/2105.06219/) (2021)
- [Cross-Modality Neuroimage Synthesis: A Survey](https://mlsanity.com/paper/2202.06997/) (2022)
- [Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2211.10867/) (2022)
- [DC-cycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Data](https://mlsanity.com/paper/2211.01293/) (2022)
- [Cross-modal tumor segmentation using generative blending augmentation and self training](https://mlsanity.com/paper/2304.01705/) (2023)
- [SPAC-Net: Synthetic Pose-aware Animal ControlNet for Enhanced Pose Estimation](https://mlsanity.com/paper/2305.17845/) (2023)
- [Deformation equivariant cross-modality image synthesis with paired non-aligned training data](https://mlsanity.com/paper/2208.12491/) (2022)
- [Unsupervised Cross-Modality Domain Adaptation for Vestibular Schwannoma Segmentation and Koos Grade Prediction based on Semi-Supervised Contrastive Learning](https://mlsanity.com/paper/2210.04255/) (2022)
- [Multi-view Cross-Modality MR Image Translation for Vestibular Schwannoma and Cochlea Segmentation](https://mlsanity.com/paper/2303.14998/) (2023)
- [Medical Image Segmentation with Domain Adaptation: A Survey](https://mlsanity.com/paper/2311.01702/) (2023)
- [StyleFlow For Content-Fixed Image to Image Translation](https://mlsanity.com/paper/2207.01909/) (2022)
- [An Overview on the Generation and Detection of Synthetic and Manipulated Satellite Images](https://mlsanity.com/paper/2209.08984/) (2022)

## Abstract (arXiv)

Unsupervised image-to-image translation is a central task in computer vision. Current translation frameworks will abandon the discriminator once the training process is completed. This paper contends a novel role of the discriminator by reusing it for encoding the images of the target domain. The proposed architecture, termed as NICE-GAN, exhibits two advantageous patterns over previous approaches: First, it is more compact since no independent encoding component is required; Second, this plug-in encoder is directly trained by the adversary loss, making it more informative and trained more effectively if a multi-scale discriminator is applied. The main issue in NICE-GAN is the coupling of translation with discrimination along the encoder, which could incur training inconsistency when we play the min-max game via GAN. To tackle this issue, we develop a decoupled training strategy by which the encoder is only trained when maximizing the adversary loss while keeping frozen otherwise. Extensive experiments on four popular benchmarks demonstrate the superior performance of NICE-GAN over state-of-the-art methods in terms of FID, KID, and also human preference. Comprehensive ablation studies are also carried out to isolate the validity of each proposed component. Our codes are available at https://github.com/alpc91/NICE-GAN-pytorch.