# Semantic Map Injected GAN Training for Image-to-Image Translation

arXiv: https://arxiv.org/abs/2112.01845 · HTML: https://mlsanity.com/paper/2112.01845/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [Generative Adversarial Minority Oversampling](https://mlsanity.com/paper/1903.09730/) (2019)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [Unsupervised Attention-guided Image to Image Translation](https://mlsanity.com/paper/1806.02311/) (2018)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence](https://mlsanity.com/paper/2005.05207/) (2020)
- [Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency](https://mlsanity.com/paper/1805.11145/) (2018)
- [Breaking the cycle -- Colleagues are all you need](https://mlsanity.com/paper/1911.10538/) (2019)
- [Example-Guided Style Consistent Image Synthesis from Semantic Labeling](https://mlsanity.com/paper/1906.01314/) (2019)
- [Sem-GAN: Semantically-Consistent Image-to-Image Translation](https://mlsanity.com/paper/1807.04409/) (2018)
- [RCCNet: An Efficient Convolutional Neural Network for Histological Routine Colon Cancer Nuclei Classification](https://mlsanity.com/paper/1810.02797/) (2018)
- [PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation](https://mlsanity.com/paper/2002.07082/) (2020)
- [A Performance Comparison of Loss Functions for Deep Face Recognition](https://mlsanity.com/paper/1901.05903/) (2019)
- [Dual CNN Models for Unsupervised Monocular Depth Estimation](https://mlsanity.com/paper/1804.06324/) (2018)
- [CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation](https://mlsanity.com/paper/1901.03554/) (2019)
- [Segmentation Guided Image-to-Image Translation with Adversarial Networks](https://mlsanity.com/paper/1901.01569/) (2019)
- [CDGAN: Cyclic Discriminative Generative Adversarial Networks for Image-to-Image Transformation](https://mlsanity.com/paper/2001.05489/) (2020)
- [Efficient High-Resolution Image-to-Image Translation using Multi-Scale Gradient U-Net](https://mlsanity.com/paper/2105.13067/) (2021)

## Abstract (arXiv)

Image-to-image translation is the recent trend to transform images from one domain to another domain using generative adversarial network (GAN). The existing GAN models perform the training by only utilizing the input and output modalities of transformation. In this paper, we perform the semantic injected training of GAN models. Specifically, we train with original input and output modalities and inject a few epochs of training for translation from input to semantic map. Lets refer the original training as the training for the translation of input image into target domain. The injection of semantic training in the original training improves the generalization capability of the trained GAN model. Moreover, it also preserves the categorical information in a better way in the generated image. The semantic map is only utilized at the training time and is not required at the test time. The experiments are performed using state-of-the-art GAN models over CityScapes and RGB-NIR stereo datasets. We observe the improved performance in terms of the SSIM, FID and KID scores after injecting semantic training as compared to original training.