mlsanityData to 14 Oct 2026

Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable

arXiv 2405.20272 ↗First posted 30 May 2024cs.LG cs.CRcited by 25

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning is motivated by desire for data autonomy: a person can request to have their data's influence removed from deployed models, and those models should be updated as if they were retrained without the person's data. We show that, counter-intuitively, these updates expose individuals to high-accuracy reconstruction attacks which allow the attacker to recover their data in its entirety, even when the original models are so simple that privacy risk might not otherwise have been a concern. We show how to mount a near-perfect attack on the deleted data point from linear regression models. We then generalize our attack to other loss functions and architectures, and empirically demonstrate the effectiveness of our attacks across a wide range of datasets (capturing both tabular and image data). Our work highlights that privacy risk is significant even for extremely simple model classes when individuals can request deletion of their data from the model.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 31 of its 39 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

31 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning4 Apr
  2. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learning17 Oct
  3. 2022Control, Confidentiality, and the Right to be Forgotten14 Oct
  4. 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning7 Feb
  5. 2021Adaptive Machine Unlearning8 Jun

Other references · 26

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  5. 2019Machine Unlearningcited by 1,061
  6. 2019Deep Leakage from Gradientscited by 1,021
  7. 2021Membership Inference Attacks From First Principlescited by 878
  8. 2023Extracting Training Data from Diffusion Modelscited by 732
  9. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  10. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  11. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  12. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
14 more
  1. 2020iDLG: Improved Deep Leakage from Gradientscited by 382
  2. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  3. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  4. 2020When Machine Unlearning Jeopardizes Privacycited by 198
  5. 2022Reconstructing Training Data with Informed Adversariescited by 148
  6. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  7. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 122
  8. 2023Scalable Membership Inference Attacks via Quantile Regressioncited by 65
  9. 2020Formalizing Data Deletion in the Context of the Right to be Forgottencited by 50
  10. 2021Interactive Label Cleaning with Example-based Explanationscited by 49
  11. 2022Reconstructing Training Data from Model Gradient, Provablycited by 27
  12. 2021Rethinking Influence Functions of Neural Networks in the Over-parameterized Regimecited by 26
  13. 2022Confidence-Ranked Reconstruction of Census Microdata from Published Statisticscited by 16
  14. 2015Revisiting Differentially Private Regression: Lessons From Learning Theory and their Consequencescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 16; 2026: 8.

25 papers.

Sort:

By area:

  1. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlapcited by 15
  2. 2025Exact Unlearning of Finetuning Data via Model Merging at Scalecited by 14
  3. 2025Model Immunization from a Condition Number Perspectivecited by 8
  4. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verificationcited by 8
  5. 2025Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLMcited by 5
  6. 2025Exposing Privacy Risks in Graph Retrieval-Augmented Generationcited by 5
  7. 2025How Secure is Forgetting? Linking Machine Unlearning to Machine Learning Attackscited by 4
  8. 2025Unlearning Inversion Attacks for Graph Neural Networkscited by 4
  9. 2025Reminiscence Attack on Residuals: Exploiting Approximate Machine Unlearning for Privacycited by 4
  10. 2025One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgettingcited by 2
  11. 2025Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearningcited by 2
  12. 2026LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearningcited by 2
13 more
  1. 2024Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Modelscited by 1
  2. 2025WARP: Weight Teleportation for Attack-Resilient Unlearning Protocolscited by 1
  3. 2025Dual-View Inference Attack: Machine Unlearning Amplifies Privacy Exposurecited by 1
  4. 2026Less Noise, Same Certificate: Retain Sensitivity for Unlearningcited by 1
  5. 2026Auditing of Unlearning Algorithmscited by 1
  6. 2025DRAUN: An Algorithm-Agnostic Data Reconstruction Attack on Federated Unlearning Systemscited by 0
  7. 2025Poison to Detect: Detection of Targeted Overfitting in Federated Learningcited by 0
  8. 2025GraphToxin: Reconstructing Full Unlearned Graphs from Graph Unlearningcited by 0
  9. 2026Towards Reliable Testing of Machine Unlearningcited by 0
  10. 2026Retain-Neutral Surrogates for Min-Max Unlearningcited by 0
  11. 2026Form and Function: Machine Unlearning as a Problem of Misaligned Statescited by 0
  12. 2026Client-Side Probing of Deleted Ridge Statistics in Federated Unlearningcited by 0
  13. 2026When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Protecting the Undeleted in Machine Unlearning
  2. 2021Hard to Forget: Poisoning Attacks on Certified Machine Unlearning
  3. 2024Survey of Security and Data Attacks on Machine Unlearning In Financial and E-Commerce
  4. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective
  5. 2024Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning
  6. 2026Revisiting Privacy Leakage in Machine Unlearning: Membership Inference Beyond the Forgotten Set
  7. 2024Releasing Malevolence from Benevolence: The Menace of Benign Data on Machine Unlearning
  8. 2026UnlearnShield: Shielding Forgotten Privacy against Unlearning Inversion