# Control of Overfitting with Physics

arXiv: https://arxiv.org/abs/2412.10716 · HTML: https://mlsanity.com/paper/2412.10716/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 14 Dec 2024.

## Builds on (most cited first, 8 of 8)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Quantum Machine Learning](https://mlsanity.com/paper/1611.09347/) (2016)
- [Gradient descent GAN optimization is locally stable](https://mlsanity.com/paper/1706.04156/) (2017)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Training Generative Adversarial Networks by Solving Ordinary Differential Equations](https://mlsanity.com/paper/2010.15040/) (2020)
- [Towards a Theory of Evolution as Multilevel Learning](https://mlsanity.com/paper/2110.14602/) (2021)
- [Functional Space Analysis of Local GAN Convergence](https://mlsanity.com/paper/2102.04448/) (2021)

## Built on by (most cited first, 1 of 1)

- [How to explain grokking](https://mlsanity.com/paper/2412.18624/) (2024)

## Abstract (arXiv)

While there are many works on the applications of machine learning, not so many of them are trying to understand the theoretical justifications to explain their efficiency. In this work, overfitting control (or generalization property) in machine learning is explained using analogies from physics and biology. For stochastic gradient Langevin dynamics, we show that the Eyring formula of kinetic theory allows to control overfitting in the algorithmic stability approach - when wide minima of the risk function with low free energy correspond to low overfitting. For the generative adversarial network (GAN) model, we establish an analogy between GAN and the predator-prey model in biology. An application of this analogy allows us to explain the selection of wide likelihood maxima and overfitting reduction for GANs.