# Selective Forgetting of Deep Networks at a Finer Level than Samples

arXiv: https://arxiv.org/abs/2012.11849 · HTML: https://mlsanity.com/paper/2012.11849/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2020.

## Built on by (most cited first, 13 of 13)

- [Boundary Unlearning](https://mlsanity.com/paper/2303.11570/) (2023)
- [Machine Unlearning via Null Space Calibration](https://mlsanity.com/paper/2404.13588/) (2024)
- [Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images](https://mlsanity.com/paper/2404.12922/) (2024)
- [Adversarial Machine Unlearning](https://mlsanity.com/paper/2406.07687/) (2024)
- [DUCK: Distance-based Unlearning via Centroid Kinematics](https://mlsanity.com/paper/2312.02052/) (2023)
- [Label Smoothing Improves Machine Unlearning](https://mlsanity.com/paper/2406.07698/) (2024)
- [Multi-Class Unlearning for Image Classification via Weight Filtering](https://mlsanity.com/paper/2304.02049/) (2023)
- [Fairness and Robustness in Machine Unlearning](https://mlsanity.com/paper/2504.13610/) (2025)
- [Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack](https://mlsanity.com/paper/2506.01318/) (2025)
- [Face Identity Unlearning for Retrieval via Embedding Dispersion](https://mlsanity.com/paper/2512.13317/) (2025)
- [Efficient Unlearning through Maximizing Relearning Convergence Delay](https://mlsanity.com/paper/2604.09391/) (2026)
- [Signal-Guided Optimization for Machine Unlearning](https://mlsanity.com/paper/2607.11975/) (2026)
- [Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning](https://mlsanity.com/paper/2609.02018/) (2026)

## Abstract (arXiv)

Selective forgetting or removing information from deep neural networks (DNNs) is essential for continual learning and is challenging in controlling the DNNs. Such forgetting is crucial also in a practical sense since the deployed DNNs may be trained on the data with outliers, poisoned by attackers, or with leaked/sensitive information. In this paper, we formulate selective forgetting for classification tasks at a finer level than the samples' level. We specify the finer level based on four datasets distinguished by two conditions: whether they contain information to be forgotten and whether they are available for the forgetting procedure. Additionally, we reveal the need for such formulation with the datasets by showing concrete and practical situations. Moreover, we introduce the forgetting procedure as an optimization problem on three criteria; the forgetting, the correction, and the remembering term. Experimental results show that the proposed methods can make the model forget to use specific information for classification. Notably, in specific cases, our methods improved the model's accuracy on the datasets, which contains information to be forgotten but is unavailable in the forgetting procedure. Such data are unexpectedly found and misclassified in actual situations.