mlsanityData to 7 Oct 2026

Behavioral Guarantees for Proxy-Based Unlearning

arXiv 2605.10680 ↗First posted 11 May 2026cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

This paper proposes a framework generalizing recent proxy-based unlearning methods and proves theoretical guarantees about the behavior of the resulting unlearned model: upper bounds on its Kullback-Leibler divergence to the ideal posterior distribution of the retain data. We model approximate unlearning as a constrained optimization problem and interpret a family of solutions as introducing a scaled unlearning signal in the output space. The unlearning signal arises from proxies of the posterior data distributions. Its scale is adapted to the proxies to ensure the behavioral upper bounds. This framework relies on the structure of the data distributions in order to create proxies. If need be, the target serves as a teacher to distill the update in the weights. Our approach is experimentally validated over two forgetting scenarios as reaching the closest classifier to the model retrained from scratch.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 53 of its 76 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

53 of 76 references.

Closest ancestors · 29

Same direction, newest first

  1. 2026Regularized f-Divergence Kernel Tests27 Jan
  2. 2025POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse24 Nov
  3. 2025Improving Unlearning with Model Updates Probably Aligned with Gradients4 Nov
  4. 2025Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgery25 Oct
  5. 2025Ascent Fails to Forget30 Sep
  6. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective16 Jun
  7. 2025UCD: Unlearning in LLMs via Contrastive Decoding12 Jun
  8. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition13 Jun
  9. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference12 Jun
  10. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  11. 2024Corrective Machine Unlearning21 Feb
  12. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
17 more
  1. 2024Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning18 Jan
  2. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  3. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  4. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  5. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  6. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  7. 2023Boundary Unlearning21 Mar
  8. 2023Towards Unbounded Machine Unlearning20 Feb
  9. 2022A Survey of Machine Unlearning6 Sep
  10. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher17 May
  11. 2022Deep Unlearning via Randomized Conditionally Independent Hessians15 Apr
  12. 2022Towards Adversarial Evaluations for Inexact Machine Unlearning17 Jan
  13. 2021Fast Yet Effective Machine Unlearning17 Nov
  14. 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
  15. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
  16. 2021Machine Unlearning of Features and Labels26 Aug
  17. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 24

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  6. 2019Machine Unlearningcited by 1,020
  7. 2021Membership Inference Attacks From First Principlescited by 839
  8. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  9. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  10. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  11. 2023Erasing Concepts from Diffusion Modelscited by 530
  12. 2019Certified Data Removal from Machine Learning Modelscited by 512
12 more
  1. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  2. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2020Amnesiac Machine Learningcited by 305
  5. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  6. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  7. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 171
  8. 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
  9. 2020Variational Bayesian Unlearningcited by 114
  10. 2023A Survey on Federated Unlearning: Challenges, Methods, and Future Directionscited by 58
  11. 2021SSSE: Efficiently Erasing Samples from Trained Machine Learning Modelscited by 28
  12. 2026Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidancecited by 1

Built on by

Not cited yet.