mlsanityData to 7 Oct 2026

Defending against Reconstruction Attacks with R'enyi Differential Privacy

arXiv 2202.07623 ↗First posted 15 Feb 2022cs.LG cs.AI cs.CRcited by 30

Not on the map. Why?

Abstract (arXiv)

Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. It has been recently shown that simple heuristics can reconstruct data samples from language models, making this threat scenario an important aspect of model release. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget (epsilon > 8) which does not translate to meaningful guarantees. In this paper we show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature. In particular, we show that larger privacy budgets do not protect against membership inference, but can still protect extraction of rare secrets. We show experimentally that our guarantees hold against various language models, including GPT-2 finetuned on Wikitext-103.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

17 of 29 references.

References · 17

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2016Pointer Sentinel Mixture Modelscited by 2,992
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2021Membership Inference Attacks From First Principlescited by 839
  8. 2017Renyi Differential Privacycited by 643
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  10. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  11. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  12. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
5 more
  1. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  2. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  3. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  4. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  5. 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 84

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 11; 2024: 8; 2025: 3; 2026: 1.

30 papers.

Sort:

By area:

  1. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  2. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  3. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  4. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  5. 2023Bounding Training Data Reconstruction in DP-SGDcited by 42
  6. 2024Measuring memorization in language models via probabilistic extractioncited by 40
  7. 2023One-shot Empirical Privacy Estimation for Federated Learningcited by 35
  8. 2024Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenoncited by 30
  9. 2024Auditing f-Differential Privacy in One Runcited by 28
  10. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
  11. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  12. 2023Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Modelscited by 10
18 more
  1. 2022Federated Boosted Decision Trees with Differential Privacycited by 9
  2. 2023DPSUR: Accelerating Differentially Private Stochastic Gradient Descent Using Selective Update and Releasecited by 9
  3. 2024The Mosaic Memory of Large Language Modelscited by 9
  4. 2022Beyond Gradients: Exploiting Adversarial Priors in Model Inversion Attackscited by 7
  5. 2022Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Designcited by 6
  6. 2023A Note On Interpreting Canary Exposurecited by 6
  7. 2023SoK: Unintended Interactions among Machine Learning Defenses and Riskscited by 6
  8. 2024Data Reconstruction Attacks and Defenses: A Systematic Evaluationcited by 5
  9. 2022Planting and Mitigating Memorized Content in Predictive-Text Language Modelscited by 1
  10. 2023SoK: Memorisation in machine learningcited by 1
  11. 2023Reconciling AI Performance and Data Reconstruction Resilience for Medical Imagingcited by 1
  12. 2024Visual Privacy Auditing with Diffusion Modelscited by 1
  13. 2024Deep Privacy Funnel Model: From a Discriminative to a Generative Approach with an Application to Face Recognitioncited by 1
  14. 2023On the Query Complexity of Training Data Reconstruction in Private Learningcited by 0
  15. 2024Optimal Defenses Against Gradient Reconstruction Attackscited by 0
  16. 2025Training Set Reconstruction from Differentially Private Forests: How Effective is DP?cited by 0
  17. 2025DeSIA: Attribute Inference Attacks Against Limited Fixed Aggregate Statisticscited by 0
  18. 2026A Sharp Transition in Data Reconstruction under Differential Privacycited by 0