# An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator

arXiv: https://arxiv.org/abs/2110.11293 · HTML: https://mlsanity.com/paper/2110.11293/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 14 of 14)

- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [A Note on the Inception Score](https://mlsanity.com/paper/1801.01973/) (2018)
- [Invertible Conditional GANs for image editing](https://mlsanity.com/paper/1611.06355/) (2016)
- [The Conditional Analogy GAN: Swapping Fashion Articles on People Images](https://mlsanity.com/paper/1709.04695/) (2017)
- [Sobolev GAN](https://mlsanity.com/paper/1711.04894/) (2017)
- [Hierarchical Implicit Models and Likelihood-Free Variational Inference](https://mlsanity.com/paper/1702.08896/) (2017)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have emerged as useful generative models, which are capable of implicitly learning data distributions of arbitrarily complex dimensions. However, the training of GANs is empirically well-known for being highly unstable and sensitive. The loss functions of both the discriminator and generator concerning their parameters tend to oscillate wildly during training. Different loss functions have been proposed to stabilize the training and improve the quality of images generated. In this paper, we perform an empirical study on the impact of several loss functions on the performance of standard GAN models, Deep Convolutional Generative Adversarial Networks (DCGANs). We introduce a new improvement that employs a relativistic discriminator to replace the classical deterministic discriminator in DCGANs and implement a margin cosine loss function for both the generator and discriminator. This results in a novel loss function, namely Relativistic Margin Cosine Loss (RMCosGAN). We carry out extensive experiments with four datasets: CIFAR-10, MNIST, STL-10, and CAT. We compare RMCosGAN performance with existing loss functions based on two metrics: Frechet inception distance and inception score. The experimental results show that RMCosGAN outperforms the existing ones and significantly improves the quality of images generated.