mlsanityData to 7 Oct 2026

An Empirical Study of Memorization in NLP

arXiv 2203.12171 ↗First posted 23 Mar 2022cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

A recent study by Feldman (2020) proposed a long-tail theory to explain the memorization behavior of deep learning models. However, memorization has not been empirically verified in the context of NLP, a gap addressed by this work. In this paper, we use three different NLP tasks to check if the long-tail theory holds. Our experiments demonstrate that top-ranked memorized training instances are likely atypical, and removing the top-memorized training instances leads to a more serious drop in test accuracy compared with removing training instances randomly. Furthermore, we develop an attribution method to better understand why a training instance is memorized. We empirically show that our memorization attribution method is faithful, and share our interesting finding that the top-memorized parts of a training instance tend to be features negatively correlated with the class label.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 23 Mar 2022.

Builds on

21 of 30 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  7. 2017A Closer Look at Memorization in Deep Networkscited by 936
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  10. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
9 more
  1. 2020Nearest Neighbor Machine Translationcited by 209
  2. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  3. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  4. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  5. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  6. 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
  7. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  8. 2021Understanding Instance-based Interpretability of Variational Auto-Encoderscited by 21
  9. 2020Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretabilitycited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 7; 2025: 3; 2026: 3.

21 papers.

Sort:

By area:

  1. 2023Intriguing Properties of Data Attribution on Diffusion Modelscited by 47
  2. 2022Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learningcited by 36
  3. 2024Localizing Paragraph Memorization in Language Modelscited by 30
  4. 2024Causal Estimation of Memorisation Profilescited by 20
  5. 2024Memorization in deep learning: A surveycited by 14
  6. 2022Finding Memo: Extractive Memorization in Constrained Sequence Generation Taskscited by 12
  7. 2024Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Taskscited by 9
  8. 2024ROME: Memorization Insights from Text, Logits and Representationcited by 8
  9. 2024Attributing Culture-Conditioned Generations to Pretraining Corporacited by 8
  10. 2023What do larger image classifiers memorise?cited by 6
  11. 2025Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyondcited by 6
  12. 2023Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translationcited by 4
9 more
  1. 2023Recursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionalitycited by 3
  2. 2023On Retrieval Augmentation and the Limitations of Language Model Trainingcited by 2
  3. 2026Bergson: An Open Source Library for Data Attributioncited by 2
  4. 2024Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Taskscited by 1
  5. 2023Long-Tail Theory under Gaussian Mixturescited by 0
  6. 2025Estimating the Effects of Sample Training Orders for Large Language Models without Retrainingcited by 0
  7. 2025Retrieval-augmented Prompt Learning for Pre-trained Foundation Modelscited by 0
  8. 2026What do Reward Models Memorize?cited by 0
  9. 2026Don't Forget! Decomposing the Training Dynamics of Memorization in Language Modelscited by 0