# An Information Theoretic Approach to Machine Unlearning

arXiv: https://arxiv.org/abs/2402.01401 · HTML: https://mlsanity.com/paper/2402.01401/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023)
- [Machine Unlearning of Features and Labels](https://mlsanity.com/paper/2108.11577/) (2021)
- [Fast Yet Effective Machine Unlearning](https://mlsanity.com/paper/2111.08947/) (2021)
- [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022)
- [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Boundary Unlearning](https://mlsanity.com/paper/2303.11570/) (2023)
- [Zero-Shot Machine Unlearning](https://mlsanity.com/paper/2201.05629/) (2022)
- [Deep Unlearning via Randomized Conditionally Independent Hessians](https://mlsanity.com/paper/2204.07655/) (2022)
- [Spectral Bias in Practice: The Role of Function Frequency in Generalization](https://mlsanity.com/paper/2110.02424/) (2021)
- [Unlearning via Sparse Representations](https://mlsanity.com/paper/2311.15268/) (2023)

## Built on by (most cited first, 16 of 16)

- [Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening](https://mlsanity.com/paper/2402.10098/) (2024)
- [Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning](https://mlsanity.com/paper/2405.19211/) (2024)
- [A Certified Unlearning Approach without Access to Source Data](https://mlsanity.com/paper/2506.06486/) (2025)
- [An Unlearning Framework for Continual Learning](https://mlsanity.com/paper/2509.17530/) (2025)
- [CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic](https://mlsanity.com/paper/2605.18610/) (2026)
- [MUC: Machine Unlearning for Contrastive Learning with Black-box Evaluation](https://mlsanity.com/paper/2406.03603/) (2024)
- [Parallel Unlearning in Inherited Model Networks](https://mlsanity.com/paper/2408.08493/) (2024)
- [Instance-Level Difficulty: A Missing Perspective in Machine Unlearning](https://mlsanity.com/paper/2410.03043/) (2024)
- [Efficient Machine Unlearning by Model Splitting and Core Sample Selection](https://mlsanity.com/paper/2505.07026/) (2025)
- [IMU: Influence-guided Machine Unlearning](https://mlsanity.com/paper/2508.01620/) (2025)
- [Reference-Guided Machine Unlearning](https://mlsanity.com/paper/2603.11210/) (2026)
- [Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors](https://mlsanity.com/paper/2508.03245/) (2025)
- [oslash Source Models Leak What They Shouldn't nrightarrow: Unlearning Zero-Shot Transfer in Domain Adaptation Through Adversarial Optimization](https://mlsanity.com/paper/2604.08238/) (2026)
- [ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition](https://mlsanity.com/paper/2605.14309/) (2026)
- [Damnatio Memoriae: Adversarially and Selectively Forgetting Identities in the Embedding Space of Face Recognition Models](https://mlsanity.com/paper/2609.27115/) (2026)
- [Why Forget-Only Unlearning Needs Memorization](https://mlsanity.com/paper/2610.10519/) (2026)

## Abstract (arXiv)

To comply with AI and data regulations, the need to forget private or copyrighted information from trained machine learning models is increasingly important. The key challenge in unlearning is forgetting the necessary data in a timely manner, while preserving model performance. In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten. We explore unlearning from an information theoretic perspective, connecting the influence of a sample to the information gain a model receives by observing it. From this, we derive a simple but principled zero-shot unlearning method based on the geometry of the model. Our approach takes the form of minimising the gradient of a learned function with respect to a small neighbourhood around a target forget point. This induces a smoothing effect, causing forgetting by moving the boundary of the classifier. We explore the intuition behind why this approach can jointly unlearn forget samples while preserving general model performance through a series of low-dimensional experiments. We perform extensive empirical evaluation of our method over a range of contemporary benchmarks, verifying that our method is competitive with state-of-the-art performance under the strict constraints of zero-shot unlearning. Code for the project can be found at https://github.com/jwf40/Information-Theoretic-Unlearning