# Toward Efficient Data-Free Unlearning

arXiv: https://arxiv.org/abs/2412.13790 · HTML: https://mlsanity.com/paper/2412.13790/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation](https://mlsanity.com/paper/2310.12508/) (2023)
- [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021)
- [Data-Free Learning of Student Networks](https://mlsanity.com/paper/1904.01186/) (2019)
- [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023)
- [Boundary Unlearning](https://mlsanity.com/paper/2303.11570/) (2023)
- [Zero-shot Knowledge Transfer via Adversarial Belief Matching](https://mlsanity.com/paper/1905.09768/) (2019)
- [Zero-Shot Knowledge Distillation in Deep Networks](https://mlsanity.com/paper/1905.08114/) (2019)
- [Data-Free Adversarial Distillation](https://mlsanity.com/paper/1912.11006/) (2019)
- [Data-Free Network Quantization With Adversarial Knowledge Distillation](https://mlsanity.com/paper/2005.04136/) (2020)
- [Contrastive Model Inversion for Data-Free Knowledge Distillation](https://mlsanity.com/paper/2105.08584/) (2021)
- [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020)
- [Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation](https://mlsanity.com/paper/2209.10359/) (2022)
- [Machine Unlearning via Null Space Calibration](https://mlsanity.com/paper/2404.13588/) (2024)
- [Small Scale Data-Free Knowledge Distillation](https://mlsanity.com/paper/2406.07876/) (2024)

## Built on by (most cited first, 4 of 4)

- [Robust Continual Unlearning against Knowledge Erosion and Forgetting Reversal](https://mlsanity.com/paper/2604.19108/) (2026)
- [Task-Agnostic Federation over Decentralized Data: Research Landscape and Visions](https://mlsanity.com/paper/2503.03140/) (2025)
- [Machine Unlearning for Streaming Forgetting](https://mlsanity.com/paper/2507.15280/) (2025)
- [SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs](https://mlsanity.com/paper/2606.09868/) (2026)

## Abstract (arXiv)

Machine unlearning without access to real data distribution is challenging. The existing method based on data-free distillation achieved unlearning by filtering out synthetic samples containing forgetting information but struggled to distill the retaining-related knowledge efficiently. In this work, we analyze that such a problem is due to over-filtering, which reduces the synthesized retaining-related information. We propose a novel method, Inhibited Synthetic PostFilter (ISPF), to tackle this challenge from two perspectives: First, the Inhibited Synthetic, by reducing the synthesized forgetting information; Second, the PostFilter, by fully utilizing the retaining-related information in synthesized samples. Experimental results demonstrate that the proposed ISPF effectively tackles the challenge and outperforms existing methods.