# Spectral Regularization for Combating Mode Collapse in GANs

arXiv: https://arxiv.org/abs/1908.10999 · HTML: https://mlsanity.com/paper/1908.10999/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2019.

## Built on by (most cited first, 30 of 33)

- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2109.02235/) (2021)
- [Underwater Light Field Retention : Neural Rendering for Underwater Imaging](https://mlsanity.com/paper/2203.11006/) (2022)
- [GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting](https://mlsanity.com/paper/2205.07162/) (2022)
- [A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution](https://mlsanity.com/paper/2111.08685/) (2021)
- [Manifold Learning Benefits GANs](https://mlsanity.com/paper/2112.12618/) (2021)
- [Improving GANs for Long-Tailed Data through Group Spectral Regularization](https://mlsanity.com/paper/2208.09932/) (2022)
- [Generating Synthetic Clinical Data that Capture Class Imbalanced Distributions with Generative Adversarial Networks: Example using Antiretroviral Therapy for HIV](https://mlsanity.com/paper/2208.08655/) (2022)
- [Synthetic Health-related Longitudinal Data with Mixed-type Variables Generated using Diffusion Models](https://mlsanity.com/paper/2303.12281/) (2023)
- [Inexpensive High Fidelity Melt Pool Models in Additive Manufacturing Using Generative Deep Diffusion](https://mlsanity.com/paper/2311.16168/) (2023)
- [Towards Practical Control of Singular Values of Convolutional Layers](https://mlsanity.com/paper/2211.13771/) (2022)
- [Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning](https://mlsanity.com/paper/2310.18209/) (2023)
- [Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting](https://mlsanity.com/paper/2401.10564/) (2024)
- [DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation](https://mlsanity.com/paper/2111.09267/) (2021)
- [Spectral Tensor Train Parameterization of Deep Learning Layers](https://mlsanity.com/paper/2103.04217/) (2021)
- [LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real World](https://mlsanity.com/paper/2409.19617/) (2024)
- [LeDiFlow: Learned Distribution-guided Flow Matching to Accelerate Image Generation](https://mlsanity.com/paper/2505.20723/) (2025)
- [Optimal Latent Vector Alignment for Unsupervised Domain Adaptation in Medical Image Segmentation](https://mlsanity.com/paper/2106.08188/) (2021)
- [Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers](https://mlsanity.com/paper/2205.06305/) (2022)
- [OrthoReg: Improving Graph-regularized MLPs via Orthogonality Regularization](https://mlsanity.com/paper/2302.00109/) (2023)
- [Incomplete Multi-view Clustering via Diffusion Completion](https://mlsanity.com/paper/2305.11489/) (2023)
- [ComFusion: Personalized Subject Generation in Multiple Specific Scenes From Single Image](https://mlsanity.com/paper/2402.11849/) (2024)
- [CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization](https://mlsanity.com/paper/2404.00521/) (2024)
- [Qualitative Data Augmentation for Performance Prediction in VLSI circuits](https://mlsanity.com/paper/2302.07566/) (2023)
- [Deep Learning based Optical Image Super-Resolution via Generative Diffusion Models for Layerwise in-situ LPBF Monitoring](https://mlsanity.com/paper/2409.13171/) (2024)
- [VARGAN: Variance Enforcing Network Enhanced GAN](https://mlsanity.com/paper/2109.02117/) (2021)
- [SIRe-Networks: Convolutional Neural Networks Architectural Extension for Information Preservation via Skip/Residual Connections and Interlaced Auto-Encoders](https://mlsanity.com/paper/2110.02776/) (2021)
- [Distribution Fitting for Combating Mode Collapse in Generative Adversarial Networks](https://mlsanity.com/paper/2212.01521/) (2022)
- [Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow](https://mlsanity.com/paper/2307.01879/) (2023)

## Abstract (arXiv)

Despite excellent progress in recent years, mode collapse remains a major unsolved problem in generative adversarial networks (GANs).In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. Theoretical analysis shows that the optimal solution to the discriminator has a strong relationship to the spectral distributions of the weight matrix.Therefore, we monitor the spectral distribution in the discriminator of spectral normalized GANs (SN-GANs), and discover a phenomenon which we refer to as spectral collapse, where a large number of singular values of the weight matrices drop dramatically when mode collapse occurs. We show that there are strong evidence linking mode collapse to spectral collapse; and based on this link, we set out to tackle spectral collapse as a surrogate of mode collapse. We have developed a spectral regularization method where we compensate the spectral distributions of the weight matrices to prevent them from collapsing, which in turn successfully prevents mode collapse in GANs. We provide theoretical explanations for why SR-GANs are more stable and can provide better performances than SN-GANs. We also present extensive experimental results and analysis to show that SR-GANs not only always outperform SN-GANs but also always succeed in combating mode collapse where SN-GANs fail. The code is available at https://github.com/max-liu-112/SRGANs-Spectral-Regularization-GANs-.