mlsanityData to 7 Oct 2026

A spin-glass model for the loss surfaces of generative adversarial networks

arXiv 2101.02524 ↗First posted 7 Jan 2021math-ph cond-mat.dis-nn cond-mat.stat-mechcited by 4

Not on the map. Why?

Abstract (arXiv)

We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 68 references that are in the map. First posted on arXiv 7 Jan 2021.

Builds on

26 of 68 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  7. 2018Self-Attention Generative Adversarial Networkscited by 809
  8. 2017Wasserstein GANcited by 635
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  10. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  11. 2014The Loss Surfaces of Multilayer Networkscited by 278
  12. 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 278
14 more
  1. 2016Coupled Generative Adversarial Networkscited by 216
  2. 2017Generalization in Deep Learningcited by 146
  3. 2018An Alternative View: When Does SGD Escape Local Minima?cited by 115
  4. 2019Asymmetric Valleys: Beyond Sharp and Flat Local Minimacited by 90
  5. 2019Shaping the learning landscape in neural networks around wide flat minimacited by 47
  6. 2018Comparing Dynamics: Deep Neural Networks versus Glassy Systemscited by 42
  7. 2017The landscape of the spiked tensor modelcited by 34
  8. 2019Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Modelscited by 28
  9. 2019Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Modelcited by 25
  10. 2018Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitionscited by 23
  11. 2020Flatness is a False Friendcited by 19
  12. 2019Landscape Complexity for the Empirical Risk of Generalized Linear Modelscited by 15
  13. 2020Beyond Random Matrix Theory for Deep Networkscited by 11
  14. 2020The Loss Surfaces of Neural Networks with General Activation Functionscited by 7

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 0; 2024: 1.

4 papers.

  1. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  2. 2024The Persistence of Neural Collapse Despite Low-Rank Biascited by 12
  3. 2021Appearance of Random Matrix Theory in Deep Learningcited by 7
  4. 2022Universal characteristics of deep neural network loss surfaces from random matrix theorycited by 4