# Contrastive Unlearning: A Contrastive Approach to Machine Unlearning

arXiv: https://arxiv.org/abs/2401.10458 · HTML: https://mlsanity.com/paper/2401.10458/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Fast Yet Effective Machine Unlearning](https://mlsanity.com/paper/2111.08947/) (2021)
- [Model Sparsity Can Simplify Machine Unlearning](https://mlsanity.com/paper/2304.04934/) (2023)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [Boundary Unlearning](https://mlsanity.com/paper/2303.11570/) (2023)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Efficient Training of Visual Transformers with Small Datasets](https://mlsanity.com/paper/2106.03746/) (2021)
- [Understanding Membership Inferences on Well-Generalized Learning Models](https://mlsanity.com/paper/1802.04889/) (2018)
- [Deep Unlearning via Randomized Conditionally Independent Hessians](https://mlsanity.com/paper/2204.07655/) (2022)
- [Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers](https://mlsanity.com/paper/2301.11578/) (2023)
- [Dissecting Supervised Contrastive Learning](https://mlsanity.com/paper/2102.08817/) (2021)

## Built on by (most cited first, 15 of 15)

- [Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocols](https://mlsanity.com/paper/2503.06991/) (2025)
- [RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models](https://mlsanity.com/paper/2603.14819/) (2026)
- [TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents](https://mlsanity.com/paper/2404.12530/) (2024)
- [Contrastive Representation Shaping for LLM Unlearning](https://mlsanity.com/paper/2601.22028/) (2026)
- [Erase at the Core: Representation Unlearning for Machine Unlearning](https://mlsanity.com/paper/2602.05375/) (2026)
- [CoUn: Empowering Machine Unlearning via Contrastive Learning](https://mlsanity.com/paper/2509.16391/) (2025)
- [WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models](https://mlsanity.com/paper/2408.09459/) (2024)
- [Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearning](https://mlsanity.com/paper/2510.00125/) (2025)
- [MUC: Machine Unlearning for Contrastive Learning with Black-box Evaluation](https://mlsanity.com/paper/2406.03603/) (2024)
- [FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model](https://mlsanity.com/paper/2502.01472/) (2025)
- [Face Identity Unlearning for Retrieval via Embedding Dispersion](https://mlsanity.com/paper/2512.13317/) (2025)
- [Erased, but Not Gone: Output Forgetting Is Not True Forgetting](https://mlsanity.com/paper/2606.25001/) (2026)
- [Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarios](https://mlsanity.com/paper/2509.23895/) (2025)
- [FedSGT: Exact Federated Unlearning via Sequential Group-based Training](https://mlsanity.com/paper/2511.23393/) (2025)
- [Supervised Contrastive Machine Unlearning of Background Bias in Sonar Image Classification with Fine-Grained Explainable AI](https://mlsanity.com/paper/2512.01291/) (2025)

## Abstract (arXiv)

Machine unlearning aims to eliminate the influence of a subset of training samples (i.e., unlearning samples) from a trained model. Effectively and efficiently removing the unlearning samples without negatively impacting the overall model performance is still challenging. In this paper, we propose a contrastive unlearning framework, leveraging the concept of representation learning for more effective unlearning. It removes the influence of unlearning samples by contrasting their embeddings against the remaining samples so that they are pushed away from their original classes and pulled toward other classes. By directly optimizing the representation space, it effectively removes the influence of unlearning samples while maintaining the representations learned from the remaining samples. Experiments on a variety of datasets and models on both class unlearning and sample unlearning showed that contrastive unlearning achieves the best unlearning effects and efficiency with the lowest performance loss compared with the state-of-the-art algorithms.