mlsanityData to 7 Oct 2026

Designing to Forget: Deep Semi-parametric Models for Unlearning

arXiv 2603.22870 ↗First posted 24 Mar 2026cs.CVcited by 1

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Recent advances in machine unlearning have focused on developing algorithms to remove specific training samples from a trained model. In contrast, we observe that not all models are equally easy to unlearn. Hence, we introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning. SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters. Empirically, we demonstrate that SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning. Notably, on ImageNet classification, SPMs reduce the prediction gap relative to a retrained (oracle) baseline by 11% and achieve over 10× faster unlearning compared to existing approaches on parametric models. The code is available at https://github.com/amberyzheng/spmunlearning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 51 of its 65 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

51 of 65 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Open Problems in Machine Unlearning for AI Safety9 Jan
  2. 2024MUNBa: Machine Unlearning via Nash Bargaining23 Nov
  3. 2024Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement29 Sep
  4. 2024Machine Unlearning: A Comprehensive Survey13 May
  5. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  6. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May
  7. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  8. 2023Boundary Unlearning21 Mar
  9. 2022Deep Regression Unlearning15 Oct
  10. 2022Towards Adversarial Evaluations for Inexact Machine Unlearning17 Jan
  11. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
  12. 2021Machine Unlearning of Features and Labels26 Aug
3 more
  1. 2021Certifiable Machine Unlearning for Linear Models29 Jun
  2. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  3. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  8. 2017Deep Setscited by 1,394
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  10. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  11. 2023Erasing Concepts from Diffusion Modelscited by 530
  12. 2019Certified Data Removal from Machine Learning Modelscited by 512
24 more
  1. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  2. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  3. 2015Deep Kernel Learningcited by 440
  4. 2023Unified Concept Editing in Diffusion Modelscited by 377
  5. 2024The Faiss librarycited by 370
  6. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  7. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  8. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  9. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 159
  10. 2018Classification-Reconstruction Learning for Open-Set Recognitioncited by 146
  11. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  12. 2020Variational Bayesian Unlearningcited by 114
  13. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  14. 2025International AI Safety Reportcited by 64
  15. 2022PUMA: Performance Unchanged Model Augmentation for Training Data Removalcited by 56
  16. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
  17. 2018Semi-parametric Image Synthesiscited by 50
  18. 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Modelscited by 41
  19. 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
  20. 2023IMMA: Immunizing text-to-image Models against Malicious Adaptationcited by 17
  21. 2025AI Safety for Everyonecited by 14
  22. 2025Model Immunization from a Condition Number Perspectivecited by 8
  23. 2024Learning to Obstruct Few-Shot Image Classification over Restricted Classescited by 5
  24. 2024Multi-concept Model Immunization through Differentiable Model Mergingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PruneForget: Joint Unlearning and Pruning of Vision Modelscited by 0