mlsanityData to 14 Oct 2026

Memorization Is Not Extraction: Tight Differential-Privacy Bounds and Audit Blind Spots

arXiv 2608.27782 ↗First posted 27 Aug 2026cs.CR cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Memorization in large language models is measured through a zoo of definitions whose formal relations are unknown, and differential privacy (DP) is treated as a proxy against all of them at once. We pin down the exact DP constant for the two that carry the practical weight, counterfactual memorization and adaptive extraction, and show that they do not control each other. Under f-DP, every adaptive extraction protocol with list budget m succeeds with probability at most 1-f(kappa) for the oblivious baseline kappa, and the bound is tight on a dense set of baselines: DP uniformly controls extraction exactly up to a threshold in how well the secret can be guessed a priori. Min-entropy certifies that baseline distribution-free, since H_∞≥εlog2 e+log2(m/τ) holds extraction below a risk level τ≤1/2 under pure ε-DP for every prior, and is exact on uniform priors. On the memorization side, f-DP caps the counterfactual memorization of any bounded score at an advantage functional η(f), equal to tanh(ε/2) under pure DP; for k≥2 duplicated copies the naive εmapsto kε bound tanh(kε/2) is unattainable, the exact constant being a closed-form staircase attained by geometric noisy counting. That cap is attained inside the local score class used in practice, and it is there that the two measures separate: one mechanism is memorized yet unextractable, another fully extractable yet exactly invisible to every loss-based score. The two-sided blind spot this opens for loss-based auditing and unlearning verification survives on billion-parameter models: a reserved-trigger release is recovered verbatim from one prompt while the audits practitioners deploy certify it clean.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

46 of 57 references.

References · 46

  1. 2016Deep Learning with Differential Privacycited by 2,728
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  5. 2019Machine Unlearningcited by 1,061
  6. 2021Membership Inference Attacks From First Principlescited by 878
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  9. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  10. 2017Renyi Differential Privacycited by 650
  11. 2019Certified Data Removal from Machine Learning Modelscited by 535
  12. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
34 more
  1. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  2. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  3. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  4. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  5. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  6. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
  7. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  8. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  9. 2013The Composition Theorem for Differential Privacycited by 297
  10. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  11. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  12. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  13. 2021Adaptive Machine Unlearningcited by 176
  14. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  15. 2022Reconstructing Training Data with Informed Adversariescited by 148
  16. 2023Emergent and Predictable Memorization in Large Language Modelscited by 148
  17. 2023Privacy Auditing with One (1) Training Runcited by 123
  18. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  19. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 104
  20. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 103
  21. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 100
  22. 2023On Provable Copyright Protection for Generative Modelscited by 98
  23. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 84
  24. 2022Bounding Training Data Reconstruction in Private (Deep) Learningcited by 46
  25. 2023Bounding Training Data Reconstruction in DP-SGDcited by 46
  26. 2024Measuring memorization in language models via probabilistic extractioncited by 41
  27. 2024Undesirable Memorization in Large Language Models: A Surveycited by 29
  28. 2023Tight Bounds for Machine Unlearning via Differential Privacycited by 18
  29. 2023Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacycited by 15
  30. 2025Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacycited by 15
  31. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigationcited by 10
  32. 2024Beyond the Calibration Point: Mechanism Comparison in Differential Privacycited by 9
  33. 2025The Sample Complexity of Membership Inference and Privacy Auditingcited by 4
  34. 2026Beyond Indistinguishability: Measuring Extraction Risk in LLM APIscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
  2. 2026Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization
  3. 2024Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language models
  4. 2022Memorization in NLP Fine-tuning Methods
  5. 2025Empirical Calibration and Metric Differential Privacy in Language Models
  6. 2025Privacy Auditing of Large Language Models
  7. 2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models
  8. 2026Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects