# Neural Network Memorization Dissection

arXiv: https://arxiv.org/abs/1911.09537 · HTML: https://mlsanity.com/paper/1911.09537/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [Memorization in deep learning: A survey](https://mlsanity.com/paper/2406.03880/) (2024)
- [Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation](https://mlsanity.com/paper/2112.08798/) (2021)
- [What do larger image classifiers memorise?](https://mlsanity.com/paper/2310.05337/) (2023)
- [Memorization in Deep Neural Networks: Does the Loss Function matter?](https://mlsanity.com/paper/2107.09957/) (2021)
- [Leveraging Unlabeled Data to Track Memorization](https://mlsanity.com/paper/2212.04461/) (2022)
- [Explainability and Robustness of Deep Visual Classification Models](https://mlsanity.com/paper/2301.01343/) (2023)
- [EMP: Enhance Memory in Data Pruning](https://mlsanity.com/paper/2408.16031/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) can easily fit a random labeling of the training data with zero training error. What is the difference between DNNs trained with random labels and the ones trained with true labels? Our paper answers this question with two contributions. First, we study the memorization properties of DNNs. Our empirical experiments shed light on how DNNs prioritize the learning of simple input patterns. In the second part, we propose to measure the similarity between what different DNNs have learned and memorized. With the proposed approach, we analyze and compare DNNs trained on data with true labels and random labels. The analysis shows that DNNs have textitOne way to Learn and textitN ways to Memorize. We also use gradient information to gain an understanding of the analysis results.