# Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten

arXiv: https://arxiv.org/abs/2202.13585 · HTML: https://mlsanity.com/paper/2202.13585/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 21 of 21)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://mlsanity.com/paper/1902.03932/) (2019)
- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [On the Accuracy of Influence Functions for Measuring Group Effects](https://mlsanity.com/paper/1905.13289/) (2019)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting](https://mlsanity.com/paper/2010.06772/) (2020)
- [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020)
- [Deep Neural Network Based Malware Detection Using Two Dimensional Binary Program Features](https://mlsanity.com/paper/1508.03096/) (2015)
- [GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection](https://mlsanity.com/paper/1903.06661/) (2019)
- [URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection](https://mlsanity.com/paper/1802.03162/) (2018)
- [MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models (Extended Version)](https://mlsanity.com/paper/1711.07477/) (2017)
- [Bayesian Inference Forgetting](https://mlsanity.com/paper/2101.06417/) (2021)
- [Know Your Phish: Novel Techniques for Detecting Phishing Sites and their Targets](https://mlsanity.com/paper/1510.06501/) (2015)

## Built on by (most cited first, 25 of 25)

- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Machine Unlearning: Solutions and Challenges](https://mlsanity.com/paper/2308.07061/) (2023)
- [Fast Model Debias with Machine Unlearning](https://mlsanity.com/paper/2310.12560/) (2023)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning](https://mlsanity.com/paper/2401.10371/) (2024)
- [Dissecting Language Models: Machine Unlearning via Selective Pruning](https://mlsanity.com/paper/2403.01267/) (2024)
- [Evaluating Machine Unlearning via Epistemic Uncertainty](https://mlsanity.com/paper/2208.10836/) (2022)
- [Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection](https://mlsanity.com/paper/2312.04095/) (2023)
- [Machine Unlearning: A Comprehensive Survey](https://mlsanity.com/paper/2405.07406/) (2024)
- [Efficiently Forgetting What You Have Learned in Graph Representation Learning via Projection](https://mlsanity.com/paper/2302.08990/) (2023)
- [Ten Challenging Problems in Federated Foundation Models](https://mlsanity.com/paper/2502.12176/) (2025)
- [Contrastive Unlearning: A Contrastive Approach to Machine Unlearning](https://mlsanity.com/paper/2401.10458/) (2024)
- [An Introduction to Machine Unlearning](https://mlsanity.com/paper/2209.00939/) (2022)
- [Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks](https://mlsanity.com/paper/2309.14054/) (2023)
- [Audit to Forget: A Unified Method to Revoke Patients' Private Data in Intelligent Healthcare](https://mlsanity.com/paper/2302.09813/) (2023)
- [Towards Reliable Forgetting: A Survey on Machine Unlearning Verification](https://mlsanity.com/paper/2506.15115/) (2025)
- [Partially Blinded Unlearning: Class Unlearning for Deep Networks a Bayesian Perspective](https://mlsanity.com/paper/2403.16246/) (2024)
- [GENIU: A Restricted Data Access Unlearning for Imbalanced Data](https://mlsanity.com/paper/2406.07885/) (2024)
- [ReVeil: Unconstrained Concealed Backdoor Attack on Deep Neural Networks using Machine Unlearning](https://mlsanity.com/paper/2502.11687/) (2025)
- ["Forgetting" in Machine Learning and Beyond: A Survey](https://mlsanity.com/paper/2405.20620/) (2024)
- [SMS: Self-supervised Model Seeding for Verification of Machine Unlearning](https://mlsanity.com/paper/2509.25613/) (2025)
- [SCU: An Efficient Machine Unlearning Scheme for Deep Learning Enabled Semantic Communications](https://mlsanity.com/paper/2502.19785/) (2025)
- [Approximate Machine Unlearning through Manifold Representation Forgetting Guided by Self Mode Connectivity](https://mlsanity.com/paper/2605.22871/) (2026)
- [How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning](https://mlsanity.com/paper/2606.02119/) (2026)
- [Generated Images Are Easier to Forget: A Machine Unlearning Perspective for Synthetic Image Detection](https://mlsanity.com/paper/2608.00716/) (2026)

## Abstract (arXiv)

As the use of machine learning (ML) models is becoming increasingly popular in many real-world applications, there are practical challenges that need to be addressed for model maintenance. One such challenge is to 'undo' the effect of a specific subset of dataset used for training a model. This specific subset may contain malicious or adversarial data injected by an attacker, which affects the model performance. Another reason may be the need for a service provider to remove data pertaining to a specific user to respect the user's privacy. In both cases, the problem is to 'unlearn' a specific subset of the training data from a trained model without incurring the costly procedure of retraining the whole model from scratch. Towards this goal, this paper presents a Markov chain Monte Carlo-based machine unlearning (MCU) algorithm. MCU helps to effectively and efficiently unlearn a trained model from subsets of training dataset. Furthermore, we show that with MCU, we are able to explain the effect of a subset of a training dataset on the model prediction. Thus, MCU is useful for examining subsets of data to identify the adversarial data to be removed. Similarly, MCU can be used to erase the lineage of a user's personal data from trained ML models, thus upholding a user's "right to be forgotten". We empirically evaluate the performance of our proposed MCU algorithm on real-world phishing and diabetes datasets. Results show that MCU can achieve a desirable performance by efficiently removing the effect of a subset of training dataset and outperform an existing algorithm that utilizes the remaining dataset.