mlsanityData to 7 Oct 2026

Statistical Mechanics of Min-Max Problems

arXiv 2409.06053 ↗First posted 9 Sep 2024cs.LG cond-mat.dis-nn math.STcited by 1

Not on the map. Why?

Abstract (arXiv)

Min-max optimization problems, also known as saddle point problems, have attracted significant attention due to their applications in various fields, such as fair beamforming, generative adversarial networks (GANs), and adversarial learning. However, understanding the properties of these min-max problems has remained a substantial challenge. This study introduces a statistical mechanical formalism for analyzing the equilibrium values of min-max problems in the high-dimensional limit, while appropriately addressing the order of operations for min and max. As a first step, we apply this formalism to bilinear min-max games and simple GANs, deriving the relationship between the amount of training data and generalization error and indicating the optimal ratio of fake to real data for effective learning. This formalism provides a groundwork for a deeper theoretical analysis of the equilibrium properties in various machine learning methods based on min-max problems and encourages the development of new algorithms and architectures.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 51 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

26 of 51 references.

References · 26

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  7. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  8. 2016Generative Adversarial Text to Image Synthesiscited by 735
  9. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  10. 2017Training GANs with Optimismcited by 217
  11. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  12. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
14 more
  1. 2020Generalisation error in learning with random features and the hidden manifold modelcited by 126
  2. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  3. 2018Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networkscited by 68
  4. 2018Negative Momentum for Improved Game Dynamicscited by 59
  5. 2018The committee machine: Computational to statistical gaps in learning a two-layers neural networkcited by 58
  6. 2020Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimizationcited by 51
  7. 2022The dynamics of representation learning in shallow, non-linear autoencoderscited by 27
  8. 2018Thermodynamics of Restricted Boltzmann Machines and related learning dynamicscited by 25
  9. 2023High-dimensional Asymptotics of Denoising Autoencoderscited by 19
  10. 2015Phase Transitions in Sparse PCAcited by 15
  11. 2018A Solvable High-Dimensional Model of GANcited by 13
  12. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  13. 2023High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curvecited by 5
  14. 2019Phase transition in PCA with missing data: Reduced signal-to-noise ratio, not sample size!cited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024High-dimensional learning of narrow neural networkscited by 12