mlsanityData to 7 Oct 2026

Gradient Surgery for One-shot Unlearning on Generative Model

arXiv 2307.04550 ↗First posted 10 Jul 2023cs.LG cs.AIcited by 11

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Recent regulation on right-to-be-forgotten emerges tons of interest in unlearning pre-trained machine learning models. While approximating a straightforward yet expensive approach of retrain-from-scratch, recent machine unlearning methods unlearn a sample by updating weights to remove its influence on the weight parameters. In this paper, we introduce a simple yet effective approach to remove a data influence on the deep generative model. Inspired by works in multi-task learning, we propose to manipulate gradients to regularize the interplay of influence among samples by projecting gradients onto the normal plane of the gradients to be retained. Our work is agnostic to statistics of the removal samples, outperforming existing baselines while providing theoretical analysis for the first time in unlearning a generative model.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 21 of its 30 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

21 of 30 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  6. 2021Variational Diffusion Modelscited by 1,145
  7. 2019Machine Unlearningcited by 1,020
  8. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  9. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  10. 2019Certified Data Removal from Machine Learning Modelscited by 512
  11. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  12. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 171
9 more
  1. 2021Adaptive Machine Unlearningcited by 164
  2. 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
  3. 2022Continual Learning and Private Unlearningcited by 124
  4. 2013Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networkscited by 84
  5. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 78
  6. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  7. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 50
  8. 2023Regularizing Second-Order Influences for Continual Learningcited by 21
  9. 2022Knowledge Removal in Sampling-based Bayesian Inferencecited by 20

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 5; 2025: 5.

11 papers.

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  2. 2024Machine Unlearning for Image-to-Image Generative Modelscited by 47
  3. 2024Boosting Alignment for Post-Unlearning Text-to-Image Generative Modelscited by 15
  4. 2024On the Limitations and Prospects of Machine Unlearning for Generative AIcited by 12
  5. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  6. 2025Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgerycited by 5
  7. 2024Moderating the Generalization of Score-based Generative Modelcited by 4
  8. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  9. 2025Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retentioncited by 1
  10. 2025UNO: Unlearning via Orthogonalization in Generative modelscited by 0
  11. 2025Controllable Machine Unlearning via Gradient Pivotingcited by 0