# Machine Unlearning via Information Theoretic Regularization

arXiv: https://arxiv.org/abs/2502.05684 · HTML: https://mlsanity.com/paper/2502.05684/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- First posted on arXiv 8 Feb 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Built on by (most cited first, 6 of 6)

- [Forgetting-MarI: LLM Unlearning via Marginal Information Regularization](https://mlsanity.com/paper/2511.11914/) (2025)
- [Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors](https://mlsanity.com/paper/2508.03245/) (2025)
- [MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines](https://mlsanity.com/paper/2510.26230/) (2025)
- [Separation-Utility Pareto Frontier: An Information-Theoretic Characterization](https://mlsanity.com/paper/2602.04408/) (2026)
- [Correlation-Guided Fast Machine Unlearning via Hessian Analysis](https://mlsanity.com/paper/2609.12620/) (2026)
- [One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning](https://mlsanity.com/paper/2610.03952/) (2026)

## Abstract (arXiv)

How can we effectively remove or ``unlearn'' undesirable information, such as specific features or the influence of individual data points, from a learning outcome while minimizing utility loss and ensuring rigorous guarantees? We introduce a unified mathematical framework based on information-theoretic regularization to address both data-point unlearning and feature unlearning. For data-point unlearning, we introduce the Marginal Unlearning Principle, an auditable and provable framework. Moreover, we provide an information-theoretic unlearning definition based on the proposed principle and provable guarantees on sufficiency and necessity of marginal unlearning. We then show that the proposed framework provides a natural solution to the marginal unlearning problem and yields auditable high-probability marginal-unlearning guarantees. For feature unlearning, the framework applies to deep learning with flexible training objectives. By combining flexibility in learning objectives with simplicity in regularization design, our approach is highly adaptable and practical for a wide range of machine learning and AI applications. From a mathematical perspective, we provide a unified analytic solution to the optimal feature unlearning problem with a variety of information-theoretic training objectives. Our theoretical analysis reveals intriguing connections between machine unlearning, information theory, optimal transport, and extremal sigma algebras. Numerical simulations support our theoretical findings.