# Less Noise, Same Certificate: Retain Sensitivity for Unlearning

arXiv: https://arxiv.org/abs/2603.03172 · HTML: https://mlsanity.com/paper/2603.03172/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 27 of 27)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [When Machine Unlearning Jeopardizes Privacy](https://mlsanity.com/paper/2005.02205/) (2020)
- [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Zero-Shot Machine Unlearning](https://mlsanity.com/paper/2201.05629/) (2022)
- [Forget Unlearning: Towards True Data-Deletion in Machine Learning](https://mlsanity.com/paper/2210.08911/) (2022)
- [Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning](https://mlsanity.com/paper/2401.10371/) (2024)
- [Towards Certified Unlearning for Deep Neural Networks](https://mlsanity.com/paper/2408.00920/) (2024)
- [Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2406.16257/) (2024)
- [Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable](https://mlsanity.com/paper/2405.20272/) (2024)
- [Avoiding Copyright Infringement via Large Language Model Unlearning](https://mlsanity.com/paper/2406.10952/) (2024)
- [The Utility and Complexity of in- and out-of-Distribution Machine Unlearning](https://mlsanity.com/paper/2412.09119/) (2024)
- [Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions](https://mlsanity.com/paper/2409.09778/) (2024)
- [Tight Bounds for Machine Unlearning via Differential Privacy](https://mlsanity.com/paper/2309.00886/) (2023)
- [Potion: Towards Poison Unlearning](https://mlsanity.com/paper/2406.09173/) (2024)
- [An Iterative Algorithm for Differentially Private k-PCA with Adaptive Noise](https://mlsanity.com/paper/2508.10879/) (2025)
- [The Space Complexity of Learning-Unlearning Algorithms](https://mlsanity.com/paper/2506.13048/) (2025)
- [Online Learning and Unlearning](https://mlsanity.com/paper/2505.08557/) (2025)

## Built on by (most cited first, 1 of 1)

- [Why Forget-Only Unlearning Needs Memorization](https://mlsanity.com/paper/2610.10519/) (2026)

## Abstract (arXiv)

Certified machine unlearning aims to provably remove the influence of a deletion set U from a model trained on a dataset S, by producing an unlearned output that is statistically indistinguishable from retraining on the retain set R := Ssetminus U. Many existing certified unlearning methods adapt techniques from Differential Privacy (DP) and add noise calibrated to global sensitivity, i.e., the worst-case output change over all adjacent datasets. We show that this DP-style calibration is often overly conservative: unlike DP, unlearning requires no indistinguishability across different retain sets, only between each deletion output and retraining on the same retain set R. Motivated by this distinction, we define retain sensitivity as the worst-case output change over deletions U' from Rcup U' while keeping R fixed. While insufficient for DP, retain sensitivity is exactly sufficient for unlearning, allowing for the same certificates with less noise. We validate these reductions in noise theoretically and empirically across several problems, including the weight of minimum spanning trees, PCA, and ERM. Finally, we refine the analysis of two widely used certified unlearning algorithms through the lens of retain sensitivity, leveraging the regularity induced by R to further reduce noise and improve utility.