# Online Forgetting Process for Linear Regression Models

arXiv: https://arxiv.org/abs/2012.01668 · HTML: https://mlsanity.com/paper/2012.01668/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 21 of 21)

- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy](https://mlsanity.com/paper/2403.01218/) (2024)
- [Machine Unlearning: A Comprehensive Survey](https://mlsanity.com/paper/2405.07406/) (2024)
- [Deep Regression Unlearning](https://mlsanity.com/paper/2210.08196/) (2022)
- [Certified Minimax Unlearning with Generalization Rates and Deletion Capacity](https://mlsanity.com/paper/2312.10336/) (2023)
- [SIFU: Sequential Informed Federated Unlearning for Efficient and Provable Client Unlearning in Federated Optimization](https://mlsanity.com/paper/2211.11656/) (2022)
- [A General Framework to Enhance Fine-tuning-based LLM Unlearning](https://mlsanity.com/paper/2502.17823/) (2025)
- [Data Plagiarism Index: Characterizing the Privacy Risk of Data-Copying in Tabular Generative Models](https://mlsanity.com/paper/2406.13012/) (2024)
- [Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models](https://mlsanity.com/paper/2406.14855/) (2024)
- [Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression](https://mlsanity.com/paper/2509.22341/) (2025)
- [Residual Bootstrap Exploration for Stochastic Linear Bandit](https://mlsanity.com/paper/2202.11474/) (2022)
- [The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models](https://mlsanity.com/paper/2507.18725/) (2025)
- [Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization](https://mlsanity.com/paper/2106.09215/) (2021)
- [Federated Online Sparse Decision Making](https://mlsanity.com/paper/2202.13448/) (2022)
- [GENIU: A Restricted Data Access Unlearning for Imbalanced Data](https://mlsanity.com/paper/2406.07885/) (2024)
- [GReaTER: Generate Realistic Tabular data after data Enhancement and Reduction](https://mlsanity.com/paper/2503.15564/) (2025)
- [Efficient Knowledge Graph Unlearning with Zeroth-order Information](https://mlsanity.com/paper/2508.14013/) (2025)
- [DEREC-SIMPRO: unlock Language Model benefits to advance Synthesis in Data Clean Room](https://mlsanity.com/paper/2411.00879/) (2024)
- [Incentivized Exploration with Stochastic Covariates: A Two-Stage Mechanism Design for Recommender System](https://mlsanity.com/paper/2406.04374/) (2024)
- [PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks](https://mlsanity.com/paper/2505.06520/) (2025)
- [Machine Unlearning for Streaming Forgetting](https://mlsanity.com/paper/2507.15280/) (2025)

## Abstract (arXiv)

Motivated by the EU's "Right To Be Forgotten" regulation, we initiate a study of statistical data deletion problems where users' data are accessible only for a limited period of time. This setting is formulated as an online supervised learning task with textitconstant memory limit. We propose a deletion-aware algorithm textttFIFD-OLS for the low dimensional case, and witness a catastrophic rank swinging phenomenon due to the data deletion operation, which leads to statistical inefficiency. As a remedy, we propose the textttFIFD-Adaptive Ridge algorithm with a novel online regularization scheme, that effectively offsets the uncertainty from deletion. In theory, we provide the cumulative regret upper bound for both online forgetting algorithms. In the experiment, we showed textttFIFD-Adaptive Ridge outperforms the ridge regression algorithm with fixed regularization level, and hopefully sheds some light on more complex statistical models.