# A Semi-Paired Approach For Label-to-Image Translation

arXiv: https://arxiv.org/abs/2306.13585 · HTML: https://mlsanity.com/paper/2306.13585/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 37 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 30 of 32)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [You Only Need Adversarial Supervision for Semantic Image Synthesis](https://mlsanity.com/paper/2012.04781/) (2020)
- [Unsupervised Cross-Domain Image Generation](https://mlsanity.com/paper/1611.02200/) (2016)
- [Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1809.05852/) (2018)
- [Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis](https://mlsanity.com/paper/1910.06809/) (2019)
- [One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1706.00826/) (2017)
- [TraVeLGAN: Image-to-image Translation by Transformation Vector Learning](https://mlsanity.com/paper/1902.09631/) (2019)
- [Semi-supervised Learning for Few-shot Image-to-Image Translation](https://mlsanity.com/paper/2003.13853/) (2020)
- [Transformation Consistency Regularization- A Semi-Supervised Paradigm for Image-to-Image Translation](https://mlsanity.com/paper/2007.07867/) (2020)
- [Harmonic Unpaired Image-to-image Translation](https://mlsanity.com/paper/1902.09727/) (2019)
- [Semantic Palette: Guiding Scene Generation with Class Proportions](https://mlsanity.com/paper/2106.01629/) (2021)
- [USIS: Unsupervised Semantic Image Synthesis](https://mlsanity.com/paper/2109.14715/) (2021)
- [Rethinking Spatially-Adaptive Normalization](https://mlsanity.com/paper/2004.02867/) (2020)
- [Towards Pragmatic Semantic Image Synthesis for Urban Scenes](https://mlsanity.com/paper/2305.09726/) (2023)
- [Semi-Supervised Image-to-Image Translation](https://mlsanity.com/paper/1901.08212/) (2019)

## Built on by (most cited first, 1 of 1)

- [SVS-GAN: Leveraging GANs for Semantic Video Synthesis](https://mlsanity.com/paper/2409.06074/) (2024)

## Abstract (arXiv)

Data efficiency, or the ability to generalize from a few labeled data, remains a major challenge in deep learning. Semi-supervised learning has thrived in traditional recognition tasks alleviating the need for large amounts of labeled data, yet it remains understudied in image-to-image translation (I2I) tasks. In this work, we introduce the first semi-supervised (semi-paired) framework for label-to-image translation, a challenging subtask of I2I which generates photorealistic images from semantic label maps. In the semi-paired setting, the model has access to a small set of paired data and a larger set of unpaired images and labels. Instead of using geometrical transformations as a pretext task like previous works, we leverage an input reconstruction task by exploiting the conditional discriminator on the paired data as a reverse generator. We propose a training algorithm for this shared network, and we present a rare classes sampling algorithm to focus on under-represented classes. Experiments on 3 standard benchmarks show that the proposed model outperforms state-of-the-art unsupervised and semi-supervised approaches, as well as some fully supervised approaches while using a much smaller number of paired samples.