mlsanityData to 7 Oct 2026

What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

arXiv 2008.03703 ↗First posted 9 Aug 2020cs.LG stat.MLcited by 492

Not on the map. Why?

Cited by 53 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning algorithms are well-known to have a propensity for fitting the training data very well and often fit even outliers and mislabeled data points. Such fitting requires memorization of training data labels, a phenomenon that has attracted significant research interest but has not been given a compelling explanation so far. A recent work of Feldman (2019) proposes a theoretical explanation for this phenomenon based on a combination of two insights. First, natural image and data distributions are (informally) known to be long-tailed, that is have a significant fraction of rare and atypical examples. Second, in a simple theoretical model such memorization is necessary for achieving close-to-optimal generalization error when the data distribution is long-tailed. However, no direct empirical evidence for this explanation or even an approach for obtaining such evidence were given. In this work we design experiments to test the key ideas in this theory. The experiments require estimation of the influence of each training example on the accuracy at each test example as well as memorization values of training examples. Estimating these quantities directly is computationally prohibitive but we show that closely-related subsampled influence and memorization values can be estimated much more efficiently. Our experiments demonstrate the significant benefits of memorization for generalization on several standard benchmarks. They also provide quantitative and visually compelling evidence for the theory put forth in (Feldman, 2019).

Key facts. Cited by 492 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 35; 2022: 57; 2023: 102; 2024: 128; 2025: 110; 2026: 60.

492 papers.

Sort:

By area:

  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  3. 2021Membership Inference Attacks From First Principlescited by 839
  4. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  5. 2023Extracting Training Data from Diffusion Modelscited by 723
  6. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  7. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  8. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  9. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  10. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  11. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  12. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
38 more
  1. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  2. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  3. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  4. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  5. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  6. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 146
  7. 2022Reconstructing Training Data with Informed Adversariescited by 143
  8. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  9. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  10. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  11. 2021Scaling Up Influence Functionscited by 130
  12. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  13. 2023On Memorization in Diffusion Modelscited by 122
  14. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 121
  15. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  16. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 116
  17. 2023Foundation Models and Fair Usecited by 112
  18. 2023On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigmcited by 107
  19. 2024How to Train Data-Efficient LLMscited by 91
  20. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  21. 2024What makes unlearning hard and what to do about itcited by 89
  22. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 88
  23. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 88
  24. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  25. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 83
  26. 2022Towards Adversarial Evaluations for Inexact Machine Unlearningcited by 82
  27. 2024Data Shapley in One Training Runcited by 76
  28. 2023Generalization on the Unseen, Logic Reasoning and Degree Curriculumcited by 73
  29. 2023Data Pruning via Moving-one-Sample-outcited by 67
  30. 2023Can Neural Network Memorization Be Localized?cited by 66
  31. 2024When Do We Not Need Larger Vision Models?cited by 65
  32. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  33. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  34. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  35. 2021Exploring Memorization in Adversarial Trainingcited by 55
  36. 2023OpenDataVal: a Unified Benchmark for Data Valuationcited by 55
  37. 2024Demystifying Verbatim Memorization in Large Language Modelscited by 55
  38. 2022Mixed Differential Privacy in Computer Visioncited by 53

50 most cited of 492 All 492 (JSON)