# Really Unlearned? Verifying Machine Unlearning via Influential Sample Pairs

arXiv: https://arxiv.org/abs/2406.10953 · HTML: https://mlsanity.com/paper/2406.10953/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 22 of 22)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations](https://mlsanity.com/paper/2003.02960/) (2020)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching](https://mlsanity.com/paper/2009.02276/) (2020)
- [Mixed-Privacy Forgetting in Deep Networks](https://mlsanity.com/paper/2012.13431/) (2020)
- [DeltaGrad: Rapid retraining of machine learning models](https://mlsanity.com/paper/2006.14755/) (2020)
- [Graph Unlearning](https://mlsanity.com/paper/2103.14991/) (2021)
- [Machine Unlearning for Random Forests](https://mlsanity.com/paper/2009.05567/) (2020)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Federated Unlearning via Class-Discriminative Pruning](https://mlsanity.com/paper/2110.11794/) (2021)
- [ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2102.02551/) (2021)
- [Recommendation Unlearning](https://mlsanity.com/paper/2201.06820/) (2022)
- [Black-box Dataset Ownership Verification via Backdoor Watermarking](https://mlsanity.com/paper/2209.06015/) (2022)
- [Proof of Unlearning: Definitions and Instantiation](https://mlsanity.com/paper/2210.11334/) (2022)
- [Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks](https://mlsanity.com/paper/2202.06312/) (2022)
- [Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS](https://mlsanity.com/paper/2011.06458/) (2020)

## Built on by (most cited first, 5 of 5)

- [Towards Reliable Forgetting: A Survey on Machine Unlearning Verification](https://mlsanity.com/paper/2506.15115/) (2025)
- [EFU: Enforcing Federated Unlearning via Functional Encryption](https://mlsanity.com/paper/2508.07873/) (2025)
- [Governing AI Forgetting: Auditing for Machine Unlearning Compliance](https://mlsanity.com/paper/2602.14553/) (2026)
- [Forgetting Similar Samples: Can Machine Unlearning Do it Better?](https://mlsanity.com/paper/2601.06938/) (2026)
- [SAU: Sparsity-Aware Unlearning for LLMs via Gradient Masking and Importance Redistribution](https://mlsanity.com/paper/2602.00577/) (2026)

## Abstract (arXiv)

Machine unlearning enables pre-trained models to eliminate the effects of partial training samples. Previous research has mainly focused on proposing efficient unlearning strategies. However, the verification of machine unlearning, or in other words, how to guarantee that a sample has been successfully unlearned, has been overlooked for a long time. Existing verification schemes typically rely on machine learning attack techniques, such as backdoor or membership inference attacks. As these techniques are not formally designed for verification, they are easily bypassed when an untrustworthy MLaaS undergoes rapid fine-tuning to merely meet the verification conditions, rather than executing real unlearning. In this paper, we propose a formal verification scheme, IndirectVerify, to determine whether unlearning requests have been successfully executed. We design influential sample pairs: one referred to as trigger samples and the other as reaction samples. Users send unlearning requests regarding trigger samples and use reaction samples to verify if the unlearning operation has been successfully carried out. We propose a perturbation-based scheme to generate those influential sample pairs. The objective is to perturb only a small fraction of trigger samples, leading to the reclassification of reaction samples. This indirect influence will be used for our verification purposes. In contrast to existing schemes that employ the same samples for all processes, our scheme, IndirectVerify, provides enhanced robustness, making it less susceptible to bypassing processes.