# Dual Projection Generative Adversarial Networks for Conditional Image Generation

arXiv: https://arxiv.org/abs/2108.09016 · HTML: https://mlsanity.com/paper/2108.09016/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [cGANs with Projection Discriminator](https://mlsanity.com/paper/1802.05637/) (2018)
- [Video Generation From Text](https://mlsanity.com/paper/1710.00421/) (2017)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching](https://mlsanity.com/paper/1709.01215/) (2017)
- [Robustness of Conditional GANs to Noisy Labels](https://mlsanity.com/paper/1811.03205/) (2018)
- [Robust Conditional GAN from Uncertainty-Aware Pairwise Comparisons](https://mlsanity.com/paper/1911.09298/) (2019)

## Built on by (most cited first, 8 of 8)

- [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.04489/) (2022)
- [Improving Tuning-Free Real Image Editing with Proximal Guidance](https://mlsanity.com/paper/2306.05414/) (2023)
- [AVID: Any-Length Video Inpainting with Diffusion Model](https://mlsanity.com/paper/2312.03816/) (2023)
- [Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training](https://mlsanity.com/paper/2111.01118/) (2021)
- [AE-StyleGAN: Improved Training of Style-Based Auto-Encoders](https://mlsanity.com/paper/2110.08718/) (2021)
- [Continuous Facial Motion Deblurring](https://mlsanity.com/paper/2207.06626/) (2022)
- [Class Conditional Time Series Generation with Structured Noise Space GAN](https://mlsanity.com/paper/2312.12946/) (2023)
- [GAN-Based Semantic Communication for Image Transmission in IoV](https://mlsanity.com/paper/2608.27989/) (2026)

## Abstract (arXiv)

Conditional Generative Adversarial Networks (cGANs) extend the standard unconditional GAN framework to learning joint data-label distributions from samples, and have been established as powerful generative models capable of generating high-fidelity imagery. A challenge of training such a model lies in properly infusing class information into its generator and discriminator. For the discriminator, class conditioning can be achieved by either (1) directly incorporating labels as input or (2) involving labels in an auxiliary classification loss. In this paper, we show that the former directly aligns the class-conditioned fake-and-real data distributions P(image|class) (em data matching), while the latter aligns data-conditioned class distributions P(class|image) (em label matching). Although class separability does not directly translate to sample quality and becomes a burden if classification itself is intrinsically difficult, the discriminator cannot provide useful guidance for the generator if features of distinct classes are mapped to the same point and thus become inseparable. Motivated by this intuition, we propose a Dual Projection GAN (P2GAN) model that learns to balance between em data matching and em label matching. We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals P(class|image) by minimizing their f-divergence. Experiments on a synthetic Mixture of Gaussian (MoG) dataset and a variety of real-world datasets including CIFAR100, ImageNet, and VGGFace2 demonstrate the efficacy of our proposed models.