mlsanityData to 7 Oct 2026

How much do language models memorize?

arXiv 2505.24832 ↗First posted 30 May 2025cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memorization have struggled to disentangle memorization from generalization. We formally separate memorization into two components: unintended memorization, the information a model contains about a specific dataset, and generalization, the information a model contains about the true data-generation process. When we completely eliminate generalization, we can compute the total memorization, which provides an estimate of model capacity: our measurements estimate that GPT-style models have a capacity of approximately 3.6 bits per parameter. We train language models on datasets of increasing size and observe that models memorize until their capacity fills, at which point "grokking" begins, and unintended memorization decreases as models begin to generalize. We train hundreds of transformer language models ranging from 500K to 1.5B parameters and produce a series of scaling laws relating model capacity and data size to membership inference.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 40 of its 55 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

40 of 55 references.

References · 40

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  5. 2017A Closer Look at Memorization in Deep Networkscited by 936
  6. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  12. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
28 more
  1. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  2. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  3. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  4. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  5. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  6. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  7. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  8. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
  9. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  10. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  11. 2024LLM Dataset Inference: Did you train on my dataset?cited by 107
  12. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  13. 2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 91
  14. 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
  15. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  16. 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 76
  17. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Modelscited by 65
  18. 2022Are Attribute Inference Attacks Just Imputation?cited by 48
  19. 2022Memorization in NLP Fine-tuning Methodscited by 43
  20. 2023Memorization Capacity of Multi-Head Attention in Transformerscited by 39
  21. 2024Scaling Laws for Fact Memorization of Large Language Modelscited by 33
  22. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  23. 2023A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learningcited by 32
  24. 2016Capacity and Trainability in Recurrent Neural Networkscited by 26
  25. 2025Language Models May Verbatim Complete Text They Were Not Explicitly Trained Oncited by 20
  26. 2024On the Optimal Memorization Capacity of Transformerscited by 10
  27. 2023Data-Copying in Generative Models: A Formal Frameworkcited by 9
  28. 2024Just How Flexible are Neural Networks in Practice?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 31.

53 papers.

Sort:

By area:

  1. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  2. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  3. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  4. 2025Learning Facts at Scale with Active Readingcited by 11
  5. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigationcited by 9
  6. 2025On The Fragility of Benchmark Contamination Detection in Reasoning Modelscited by 8
  7. 2025Exploiting Vocabulary Frequency Imbalance in Language Model Pre-trainingcited by 7
  8. 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impactcited by 6
  9. 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
  10. 2026WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantizationcited by 5
  11. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  12. 2026Operationalising the Superficial Alignment Hypothesis via Task Complexitycited by 4
38 more
  1. 2026Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speedscited by 4
  2. 2025Not All Bits Are Equal: Scale-Dependent Memory Optimization Strategies for Reasoning Modelscited by 3
  3. 2026Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMscited by 3
  4. 2026Unintended Memorization of Sensitive Information in Fine-Tuned Language Modelscited by 3
  5. 2026Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarkscited by 3
  6. 2025Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Studycited by 2
  7. 2025Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliancecited by 2
  8. 2025Machine Text Detectors are Membership Inference Attackscited by 2
  9. 2026Memorization Dynamics in Knowledge Distillation for Language Modelscited by 2
  10. 2026GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMscited by 2
  11. 2026Cram Less to Fit More: Training Data Pruning Improves Memorization of Factscited by 2
  12. 2025Data Compressibility Quantifies LLM Memorizationcited by 1
  13. 2025Access Paths for Efficient Ordering with Large Language Modelscited by 1
  14. 2025AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAMcited by 1
  15. 2025From Memorization to Reasoning in the Spectrum of Loss Curvaturecited by 1
  16. 2026A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Modelscited by 1
  17. 2026The Future of Facts: Tracing the Factual Generation-Verification Gapcited by 1
  18. 2025Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncodercited by 0
  19. 2025Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMscited by 0
  20. 2025The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?cited by 0
  21. 2025FiMMIA: scaling semantic perturbation-based membership inference across modalitiescited by 0
  22. 2026Semantic Rate Distortion and Posterior Design: Compute Constraints, Multimodality, and Strategic Inferencecited by 0
  23. 2026LaCy: What Small Language Models Can and Should Learn is Not Just a Question of Losscited by 0
  24. 2026Large Language Models and Book Summarization: Reading or Remembering, Which Is Better?cited by 0
  25. 2026Predicting LLM Compression Degradation from Spectral Statisticscited by 0
  26. 2026Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learningcited by 0
  27. 2026Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?cited by 0
  28. 2026Fast Unlearning at Scale via Margin Self-Correctioncited by 0
  29. 2026PromptMN: Pseudo Prompting Languagecited by 0
  30. 2026Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Trainingcited by 0
  31. 2026Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuningcited by 0
  32. 2026Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Datacited by 0
  33. 2026Enhancing SLMs for Sustainable Code Optimization in Radio-Astronomycited by 0
  34. 2026Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographiescited by 0
  35. 2026Training Large Language Models for Small-Molecule Design with Synthetic Task Scalingcited by 0
  36. 2026Specification Oraclescited by 0
  37. 2026Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Modelscited by 0
  38. 2026Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switchcited by 0

50 most cited of 53 All 53 (JSON)