mlsanityData to 7 Oct 2026

Contrastive Unlearning: A Contrastive Approach to Machine Unlearning

arXiv 2401.10458 ↗First posted 19 Jan 2024cs.LG cs.CRcited by 14

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to eliminate the influence of a subset of training samples (i.e., unlearning samples) from a trained model. Effectively and efficiently removing the unlearning samples without negatively impacting the overall model performance is still challenging. In this paper, we propose a contrastive unlearning framework, leveraging the concept of representation learning for more effective unlearning. It removes the influence of unlearning samples by contrasting their embeddings against the remaining samples so that they are pushed away from their original classes and pulled toward other classes. By directly optimizing the representation space, it effectively removes the influence of unlearning samples while maintaining the representations learned from the remaining samples. Experiments on a variety of datasets and models on both class unlearning and sample unlearning showed that contrastive unlearning achieves the best unlearning effects and efficiency with the lowest performance loss compared with the state-of-the-art algorithms.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 32 of its 45 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

32 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  2. 2023Boundary Unlearning21 Mar
  3. 2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers27 Jan
  4. 2022Deep Unlearning via Randomized Conditionally Independent Hessians15 Apr
  5. 2022Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten28 Feb
  6. 2021Fast Yet Effective Machine Unlearning17 Nov
  7. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
  8. 2021Adaptive Machine Unlearning8 Jun

Other references · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2020Supervised Contrastive Learningcited by 2,803
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  10. 2019Machine Unlearningcited by 1,020
  11. 2017A Closer Look at Memorization in Deep Networkscited by 936
  12. 2021Membership Inference Attacks From First Principlescited by 839
12 more
  1. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  2. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  3. 2019Certified Data Removal from Machine Learning Modelscited by 512
  4. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  5. 2018An Empirical Model of Large-Batch Trainingcited by 268
  6. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  7. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  8. 2020Variational Bayesian Unlearningcited by 114
  9. 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
  10. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 83
  11. 2021Dissecting Supervised Contrastive Learningcited by 64
  12. 2019On the Separability of Classes with the Cross-Entropy Loss Functioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025Are We Truly Forgetting? A Critical Re-examination of Machine Unlearning Evaluation Protocolscited by 16
  2. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agentscited by 7
  3. 2025CoUn: Empowering Machine Unlearning via Contrastive Learningcited by 4
  4. 2026Contrastive Representation Shaping for LLM Unlearningcited by 4
  5. 2024WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Modelscited by 3
  6. 2026RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Modelscited by 3
  7. 2025Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearningcited by 2
  8. 2026Erase at the Core: Representation Unlearning for Machine Unlearningcited by 2
  9. 2024MUC: Machine Unlearning for Contrastive Learning with Black-box Evaluationcited by 1
  10. 2025FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Modelcited by 1
  11. 2025Face Identity Unlearning for Retrieval via Embedding Dispersioncited by 1
  12. 2025Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarioscited by 0
2 more
  1. 2025Supervised Contrastive Machine Unlearning of Background Bias in Sonar Image Classification with Fine-Grained Explainable AIcited by 0
  2. 2026Erased, but Not Gone: Output Forgetting Is Not True Forgettingcited by 0