# Generalization and Memorization: The Bias Potential Model

arXiv: https://arxiv.org/abs/2011.14269 · HTML: https://mlsanity.com/paper/2011.14269/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023)
- [On the Generalization Properties of Diffusion Models](https://mlsanity.com/paper/2311.01797/) (2023)
- [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024)
- [Generalization Error of GAN from the Discriminator's Perspective](https://mlsanity.com/paper/2107.03633/) (2021)
- [A Mathematical Framework for Learning Probability Distributions](https://mlsanity.com/paper/2212.11481/) (2022)
- [A Class of Dimension-free Metrics for the Convergence of Empirical Measures](https://mlsanity.com/paper/2104.12036/) (2021)
- [Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis](https://mlsanity.com/paper/2506.00849/) (2025)
- [Generalized Balancing Weights via Deep Neural Networks](https://mlsanity.com/paper/2211.07533/) (2022)
- [Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness](https://mlsanity.com/paper/2307.02454/) (2023)

## Abstract (arXiv)

Models for learning probability distributions such as generative models and density estimators behave quite differently from models for learning functions. One example is found in the memorization phenomenon, namely the ultimate convergence to the empirical distribution, that occurs in generative adversarial networks (GANs). For this reason, the issue of generalization is more subtle than that for supervised learning. For the bias potential model, we show that dimension-independent generalization accuracy is achievable if early stopping is adopted, despite that in the long term, the model either memorizes the samples or diverges.