# Generalization-Memorization Machines

arXiv: https://arxiv.org/abs/2207.03976 · HTML: https://mlsanity.com/paper/2207.03976/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 7 of 7)

- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Accurate Uncertainty Estimation and Decomposition in Ensemble Learning](https://mlsanity.com/paper/1911.04061/) (2019)
- [When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data](https://mlsanity.com/paper/2005.10940/) (2020)
- [Robust subspace clustering by Cauchy loss function](https://mlsanity.com/paper/1904.12274/) (2019)
- [Insensitive Stochastic Gradient Twin Support Vector Machine for Large Scale Problems](https://mlsanity.com/paper/1704.05596/) (2017)

## Built on by (most cited first, 2 of 2)

- [ResMem: Learn what you can and memorize the rest](https://mlsanity.com/paper/2302.01576/) (2023)
- [Least Squares Maximum and Weighted Generalization-Memorization Machines](https://mlsanity.com/paper/2308.16456/) (2023)

## Abstract (arXiv)

Classifying the training data correctly without over-fitting is one of the goals in machine learning. In this paper, we propose a generalization-memorization mechanism, including a generalization-memorization decision and a memory modeling principle. Under this mechanism, error-based learning machines improve their memorization abilities of training data without over-fitting. Specifically, the generalization-memorization machines (GMM) are proposed by applying this mechanism. The optimization problems in GMM are quadratic programming problems and could be solved efficiently. It should be noted that the recently proposed generalization-memorization kernel and the corresponding support vector machines are the special cases of our GMM. Experimental results show the effectiveness of the proposed GMM both on memorization and generalization.