# Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable

arXiv: https://arxiv.org/abs/2405.20272 · HTML: https://mlsanity.com/paper/2405.20272/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 30 May 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 31)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [When Machine Unlearning Jeopardizes Privacy](https://mlsanity.com/paper/2005.02205/) (2020)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Reconstructing Training Data with Informed Adversaries](https://mlsanity.com/paper/2201.04845/) (2022)
- [If Influence Functions are the Answer, Then What is the Question?](https://mlsanity.com/paper/2209.05364/) (2022)
- [Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning](https://mlsanity.com/paper/1904.01067/) (2019)
- [Scalable Membership Inference Attacks via Quantile Regression](https://mlsanity.com/paper/2307.03694/) (2023)
- [Forget Unlearning: Towards True Data-Deletion in Machine Learning](https://mlsanity.com/paper/2210.08911/) (2022)
- [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020)
- [Interactive Label Cleaning with Example-based Explanations](https://mlsanity.com/paper/2106.03922/) (2021)
- [Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning](https://mlsanity.com/paper/2404.03233/) (2024)
- [Reconstructing Training Data from Model Gradient, Provably](https://mlsanity.com/paper/2212.03714/) (2022)
- [Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime](https://mlsanity.com/paper/2112.08297/) (2021)
- [Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning](https://mlsanity.com/paper/2202.03460/) (2022)
- [Confidence-Ranked Reconstruction of Census Microdata from Published Statistics](https://mlsanity.com/paper/2211.03128/) (2022)
- [Control, Confidentiality, and the Right to be Forgotten](https://mlsanity.com/paper/2210.07876/) (2022)

## Built on by (most cited first, 25 of 25)

- [BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap](https://mlsanity.com/paper/2506.15699/) (2025)
- [Exact Unlearning of Finetuning Data via Model Merging at Scale](https://mlsanity.com/paper/2504.04626/) (2025)
- [Model Immunization from a Condition Number Perspective](https://mlsanity.com/paper/2505.23760/) (2025)
- [Towards Reliable Forgetting: A Survey on Machine Unlearning Verification](https://mlsanity.com/paper/2506.15115/) (2025)
- [Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM](https://mlsanity.com/paper/2505.24379/) (2025)
- [Exposing Privacy Risks in Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.17222/) (2025)
- [How Secure is Forgetting? Linking Machine Unlearning to Machine Learning Attacks](https://mlsanity.com/paper/2503.20257/) (2025)
- [Unlearning Inversion Attacks for Graph Neural Networks](https://mlsanity.com/paper/2506.00808/) (2025)
- [Reminiscence Attack on Residuals: Exploiting Approximate Machine Unlearning for Privacy](https://mlsanity.com/paper/2507.20573/) (2025)
- [One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting](https://mlsanity.com/paper/2507.07754/) (2025)
- [Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearning](https://mlsanity.com/paper/2510.17917/) (2025)
- [LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning](https://mlsanity.com/paper/2607.02513/) (2026)
- [Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models](https://mlsanity.com/paper/2410.03039/) (2024)
- [WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols](https://mlsanity.com/paper/2512.00272/) (2025)
- [Dual-View Inference Attack: Machine Unlearning Amplifies Privacy Exposure](https://mlsanity.com/paper/2512.16126/) (2025)
- [Less Noise, Same Certificate: Retain Sensitivity for Unlearning](https://mlsanity.com/paper/2603.03172/) (2026)
- [Auditing of Unlearning Algorithms](https://mlsanity.com/paper/2607.05898/) (2026)
- [DRAUN: An Algorithm-Agnostic Data Reconstruction Attack on Federated Unlearning Systems](https://mlsanity.com/paper/2506.01777/) (2025)
- [Poison to Detect: Detection of Targeted Overfitting in Federated Learning](https://mlsanity.com/paper/2509.11974/) (2025)
- [GraphToxin: Reconstructing Full Unlearned Graphs from Graph Unlearning](https://mlsanity.com/paper/2511.10936/) (2025)
- [Towards Reliable Testing of Machine Unlearning](https://mlsanity.com/paper/2604.16536/) (2026)
- [Retain-Neutral Surrogates for Min-Max Unlearning](https://mlsanity.com/paper/2605.05871/) (2026)
- [Form and Function: Machine Unlearning as a Problem of Misaligned States](https://mlsanity.com/paper/2605.17590/) (2026)
- [Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning](https://mlsanity.com/paper/2609.04475/) (2026)
- [When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Data](https://mlsanity.com/paper/2609.06786/) (2026)

## Abstract (arXiv)

Machine unlearning is motivated by desire for data autonomy: a person can request to have their data's influence removed from deployed models, and those models should be updated as if they were retrained without the person's data. We show that, counter-intuitively, these updates expose individuals to high-accuracy reconstruction attacks which allow the attacker to recover their data in its entirety, even when the original models are so simple that privacy risk might not otherwise have been a concern. We show how to mount a near-perfect attack on the deleted data point from linear regression models. We then generalize our attack to other loss functions and architectures, and empirically demonstrate the effectiveness of our attacks across a wide range of datasets (capturing both tabular and image data). Our work highlights that privacy risk is significant even for extremely simple model classes when individuals can request deletion of their data from the model.