# Bounding Training Data Reconstruction in Private (Deep) Learning

arXiv: https://arxiv.org/abs/2201.12383 · HTML: https://mlsanity.com/paper/2201.12383/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Reconstructing Training Data with Informed Adversaries](https://mlsanity.com/paper/2201.04845/) (2022)
- [Tempered Sigmoid Activations for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2007.14191/) (2020)
- [Measuring Data Leakage in Machine-Learning Models with Fisher Information](https://mlsanity.com/paper/2102.11673/) (2021)
- [Three Variants of Differential Privacy: Lossless Conversion and Applications](https://mlsanity.com/paper/2008.06529/) (2020)
- [That which we call private](https://mlsanity.com/paper/1908.03566/) (2019)

## Built on by (most cited first, 30 of 46)

- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Foundation Models and Fair Use](https://mlsanity.com/paper/2303.15715/) (2023)
- [Bounding Training Data Reconstruction in DP-SGD](https://mlsanity.com/paper/2302.07225/) (2023)
- [One-shot Empirical Privacy Estimation for Federated Learning](https://mlsanity.com/paper/2302.03098/) (2023)
- [Privacy-Preserving Instructions for Aligning Large Language Models](https://mlsanity.com/paper/2402.13659/) (2024)
- [Split-and-Denoise: Protect large language model inference with local differential privacy](https://mlsanity.com/paper/2310.09130/) (2023)
- [Auditing f-Differential Privacy in One Run](https://mlsanity.com/paper/2410.22235/) (2024)
- [Unleashing the Power of Randomization in Auditing Differentially Private ML](https://mlsanity.com/paper/2305.18447/) (2023)
- [Leave-one-out Distinguishability in Machine Learning](https://mlsanity.com/paper/2309.17310/) (2023)
- [Do SSL Models Have D'ej`a Vu? A Case of Unintended Memorization in Self-supervised Learning](https://mlsanity.com/paper/2304.13850/) (2023)
- [Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano](https://mlsanity.com/paper/2210.13662/) (2022)
- [Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy](https://mlsanity.com/paper/2307.03928/) (2023)
- [Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Information](https://mlsanity.com/paper/2305.04146/) (2023)
- [Information Flow Control in Machine Learning through Modular Model Architecture](https://mlsanity.com/paper/2306.03235/) (2023)
- [Gradients Look Alike: Sensitivity is Often Overestimated in DP-SGD](https://mlsanity.com/paper/2307.00310/) (2023)
- [Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design](https://mlsanity.com/paper/2211.03942/) (2022)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [Data Reconstruction Attacks and Defenses: A Systematic Evaluation](https://mlsanity.com/paper/2402.09478/) (2024)
- [Differentially Private Representation Learning via Image Captioning](https://mlsanity.com/paper/2403.02506/) (2024)
- [Sequentially Auditing Differential Privacy](https://mlsanity.com/paper/2509.07055/) (2025)
- [Defending Against Data Reconstruction Attacks in Federated Learning: An Information Theory Approach](https://mlsanity.com/paper/2403.01268/) (2024)
- [Training Data Reconstruction: Privacy due to Uncertainty?](https://mlsanity.com/paper/2412.08544/) (2024)
- [Measuring and Controlling Split Layer Privacy Leakage Using Fisher Information](https://mlsanity.com/paper/2209.10119/) (2022)
- [Scaling Private Deep Learning with Low-Rank and Sparse Gradients](https://mlsanity.com/paper/2207.02699/) (2022)
- [How Do Input Attributes Impact the Privacy Loss in Differential Privacy?](https://mlsanity.com/paper/2211.10173/) (2022)
- [Implicit Bias in Noisy-SGD: With Applications to Differentially Private Training](https://mlsanity.com/paper/2402.08344/) (2024)
- [Privacy Amplification for the Gaussian Mechanism via Bounded Support](https://mlsanity.com/paper/2403.05598/) (2024)
- [Visual Privacy Auditing with Diffusion Models](https://mlsanity.com/paper/2403.07588/) (2024)
- [Machine Learning with Privacy for Protected Attributes](https://mlsanity.com/paper/2506.19836/) (2025)
- [From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks](https://mlsanity.com/paper/2402.12861/) (2024)

## Abstract (arXiv)

Differential privacy is widely accepted as the de facto method for preventing data leakage in ML, and conventional wisdom suggests that it offers strong protection against privacy attacks. However, existing semantic guarantees for DP focus on membership inference, which may overestimate the adversary's capabilities and is not applicable when membership status itself is non-sensitive. In this paper, we derive the first semantic guarantees for DP mechanisms against training data reconstruction attacks under a formal threat model. We show that two distinct privacy accounting methods -- Renyi differential privacy and Fisher information leakage -- both offer strong semantic protection against data reconstruction attacks.