Data Deletion for Linear Regression with Noisy SGD
Not on the map. Why?
Abstract (arXiv)
In the current era of big data and machine learning, it's essential to find ways to shrink the size of training dataset while preserving the training performance to improve efficiency. However, the challenge behind it includes providing practical ways to find points that can be deleted without significantly harming the training result and suffering from problems like underfitting. We therefore present the perfect deleted point problem for 1-step noisy SGD in the classical linear regression task, which aims to find the perfect deleted point in the training dataset such that the model resulted from the deleted dataset will be identical to the one trained without deleting it. We apply the so-called signal-to-noise ratio and suggest that its value is closely related to the selection of the perfect deleted point. We also implement an algorithm based on this and empirically show the effectiveness of it in a synthetic dataset. Finally we analyze the consequences of the perfect deleted point, specifically how it affects the training performance and privacy budget, therefore highlighting its potential. This research underscores the importance of data deletion and calls for urgent need for more studies in this field.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 12 Oct 2024.
Builds on
24 of 48 references.
References · 24
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Machine Unlearningcited by 1,020
- 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
- 2019Certified Data Removal from Machine Learning Modelscited by 512
- 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
- 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
- 2020Amnesiac Machine Learningcited by 305
- 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
- 2020Approximate Data Deletion from Machine Learning Modelscited by 250
- 2020When Machine Unlearning Jeopardizes Privacycited by 181
- 2021Adaptive Machine Unlearningcited by 164
- 2020DeltaGrad: Rapid retraining of machine learning modelscited by 143
12 more
- 2023Privacy Auditing with One (1) Training Runcited by 119
- 2020Machine Unlearning: Linear Filtration for Logit-based Classifierscited by 89
- 2022Forget Unlearning: Towards True Data-Deletion in Machine Learningcited by 58
- 2020Formalizing Data Deletion in the Context of the Right to be Forgottencited by 46
- 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
- 2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 35
- 2021Differentially private inference via noisy optimizationcited by 27
- 2015Do We Need More Training Data?cited by 26
- 2023Fair Machine Unlearning: Data Removal while Mitigating Disparitiescited by 25
- 2022Beyond Uniform Lipschitz Condition in Differentially Private Optimizationcited by 19
- 2024Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Modelcited by 17
- 2024BadGD: A unified data-centric framework to identify gradient descent vulnerabilitiescited by 5
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.