# Netflix and Forget: Efficient and Exact Machine Unlearning from Bi-linear Recommendations

arXiv: https://arxiv.org/abs/2302.06676 · HTML: https://mlsanity.com/paper/2302.06676/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 23 of its 59 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 23 of 23)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [When Machine Unlearning Jeopardizes Privacy](https://mlsanity.com/paper/2005.02205/) (2020)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)
- [Fast Matrix Factorization for Online Recommendation with Implicit Feedback](https://mlsanity.com/paper/1708.05024/) (2017)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [Recommendation Unlearning](https://mlsanity.com/paper/2201.06820/) (2022)
- [Degenerate Feedback Loops in Recommender Systems](https://mlsanity.com/paper/1902.10730/) (2019)
- [Making Recommender Systems Forget: Learning and Unlearning for Erasable Recommendation](https://mlsanity.com/paper/2203.11491/) (2022)
- [Fast Differentially Private Matrix Factorization](https://mlsanity.com/paper/1505.01419/) (2015)
- [Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries](https://mlsanity.com/paper/2009.00395/) (2020)

## Built on by (most cited first, 12 of 12)

- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [BLUR: A Bi-Level Optimization Approach for LLM Unlearning](https://mlsanity.com/paper/2506.08164/) (2025)
- [MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts](https://mlsanity.com/paper/2409.11844/) (2024)
- [Towards Efficient and Effective Unlearning of Large Language Models for Recommendation](https://mlsanity.com/paper/2403.03536/) (2024)
- [A Survey of Graph Unlearning](https://mlsanity.com/paper/2310.02164/) (2023)
- [Machine Unlearning for Recommendation Systems: An Insight](https://mlsanity.com/paper/2401.10942/) (2024)
- [A Survey on Recommendation Unlearning: Fundamentals, Taxonomy, Evaluation, and Open Questions](https://mlsanity.com/paper/2412.12836/) (2024)
- [Recommendation Unlearning via Matrix Correction](https://mlsanity.com/paper/2307.15960/) (2023)
- [On the Effectiveness of Unlearning in Session-Based Recommendation](https://mlsanity.com/paper/2312.14447/) (2023)
- [Customized Retrieval-Augmented Generation with LLM for Debiasing Recommendation Unlearning](https://mlsanity.com/paper/2511.05494/) (2025)
- [Obliviate: Efficient Unlearning in Recommender Systems](https://mlsanity.com/paper/2607.22665/) (2026)

## Abstract (arXiv)

People break up, miscarry, and lose loved ones. Their online streaming and shopping recommendations, however, do not necessarily update, and may serve as unhappy reminders of their loss. When users want to renege on their past actions, they expect the recommender platforms to erase selective data at the model level. Ideally, given any specified user history, the recommender can unwind or "forget", as if the record was not part of training. To that end, this paper focuses on simple but widely deployed bi-linear models for recommendations based on matrix completion. Without incurring the cost of re-training, and without degrading the model unnecessarily, we develop Unlearn-ALS by making a few key modifications to the fine-tuning procedure under Alternating Least Squares optimisation, thus applicable to any bi-linear models regardless of the training procedure. We show that Unlearn-ALS is consistent with retraining without any model degradation and exhibits rapid convergence, making it suitable for a large class of existing recommenders.