mlsanityData to 7 Oct 2026

Emergent and Predictable Memorization in Large Language Models

arXiv 2304.11158 ↗First posted 21 Apr 2023cs.CLcited by 147

Not on the map. Why?

Cited by 25 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Memorization, or the tendency of large language models (LLMs) to output entire sequences from their training data verbatim, is a key concern for safely deploying language models. In particular, it is vital to minimize a model's memorization of sensitive datapoints such as those containing personal identifiable information (PII). The prevalence of such undesirable memorization can pose issues for model trainers, and may even require discarding an otherwise functional model. We therefore seek to predict which sequences will be memorized before a large model's full train-time by extrapolating the memorization behavior of lower-compute trial runs. We measure memorization of the Pythia model suite and plot scaling laws for forecasting memorization, allowing us to provide equi-compute recommendations to maximize the reliability (recall) of such predictions. We additionally provide further novel discoveries on the distribution of memorization scores across models and data. We release all code and data necessary to reproduce the results in this paper at https://github.com/EleutherAI/pythia

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). Builds on 49 of its 63 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

49 of 63 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2016Deep Learning with Differential Privacycited by 2,682
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
37 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  5. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  10. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  12. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  13. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  14. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  15. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  16. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  17. 2020Amnesiac Machine Learningcited by 305
  18. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  19. 2021Scaling Laws for Transfercited by 265
  20. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  21. 2022Predictability and Surprise in Large Generative Modelscited by 209
  22. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  23. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 147
  24. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  25. 2021Large-Scale Differentially Private BERTcited by 117
  26. 2023Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Clustercited by 106
  27. 2022Broken Neural Scaling Lawscited by 102
  28. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  29. 2022Benign Overfitting in Two-layer Convolutional Neural Networkscited by 90
  30. 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
  31. 2022Kernelized Concept Erasurecited by 47
  32. 2022Datasheet for the Pilecited by 41
  33. 2021A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?cited by 14
  34. 2022EleutherAI: Going Beyond "Open Science" to "Science in the Open"cited by 13
  35. 2022Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resourcescited by 11
  36. 2021Obstructing Classification via Projectioncited by 5
  37. 2017Differentially Private Distributed Learning for Language Modeling Taskscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 50; 2025: 44; 2026: 25.

147 papers.

Sort:

By area:

  1. 2025Gemma 3 Technical Reportcited by 1,686
  2. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  3. 2023Scaling Data-Constrained Language Modelscited by 391
  4. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  5. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  6. 2023Challenges and Applications of Large Language Modelscited by 251
  7. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  8. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  9. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  10. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  11. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 126
  12. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
38 more
  1. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  2. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 100
  3. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 100
  4. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  5. 2023Efficient Benchmarking of Language Modelscited by 71
  6. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  7. 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 66
  8. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
  9. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  10. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  11. 2024The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarizationcited by 49
  12. 2023Exploring Memorization in Fine-tuned Language Modelscited by 44
  13. 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 43
  14. 2024SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Modelscited by 43
  15. 2023Data Contamination Through the Lens of Timecited by 39
  16. 2023User Inference Attacks on Large Language Modelscited by 36
  17. 2024Copyright Protection in Generative AI: A Technical Perspectivecited by 35
  18. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  19. 2024Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMscited by 33
  20. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  21. 2024Large language models can be zero-shot anomaly detectors for time series?cited by 30
  22. 2024Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenoncited by 30
  23. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 29
  24. 2024Undesirable Memorization in Large Language Models: A Surveycited by 29
  25. 2023Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytellingcited by 26
  26. 2023Traces of Memorisation in Large Language Models for Codecited by 26
  27. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  28. 2024Large Language Model Distilling Medication Recommendation Modelcited by 25
  29. 2023Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournamentcited by 24
  30. 2025Privacy Auditing of Large Language Modelscited by 22
  31. 2023Tools for Verifying Neural Models' Training Datacited by 21
  32. 2024AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracycited by 21
  33. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Directioncited by 21
  34. 2025Language Models May Verbatim Complete Text They Were Not Explicitly Trained Oncited by 20
  35. 2024Quantifying Generalization Complexity for Large Language Modelscited by 18
  36. 2024Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Modelscited by 16
  37. 2023What can we learn from Data Leakage and Unlearning for Law?cited by 15
  38. 2024Code Simulation Challenges for Large Language Modelscited by 15

50 most cited of 147 All 147 (JSON)