# SMI: Statistical Membership Inference for Reliable Unlearned Model Auditing

arXiv: https://arxiv.org/abs/2602.01150 · HTML: https://mlsanity.com/paper/2602.01150/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 1 Feb 2026.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [When Machine Unlearning Jeopardizes Privacy](https://mlsanity.com/paper/2005.02205/) (2020)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy](https://mlsanity.com/paper/2403.01218/) (2024)
- [Mixture Proportion Estimation via Kernel Embedding of Distributions](https://mlsanity.com/paper/1603.02501/) (2016)
- [Backdoor Defense with Machine Unlearning](https://mlsanity.com/paper/2201.09538/) (2022)
- [Spurious Forgetting in Continual Learning of Language Models](https://mlsanity.com/paper/2501.13453/) (2025)
- [Near-optimal inference in adaptive linear regression](https://mlsanity.com/paper/2107.02266/) (2021)
- [Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness](https://mlsanity.com/paper/2506.06112/) (2025)
- [Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble](https://mlsanity.com/paper/2506.13972/) (2025)
- [Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models](https://mlsanity.com/paper/2402.11989/) (2024)
- [Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning](https://mlsanity.com/paper/2412.08559/) (2024)
- [Data Duplication: A Novel Multi-Purpose Attack Paradigm in Machine Unlearning](https://mlsanity.com/paper/2501.16663/) (2025)
- [Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective](https://mlsanity.com/paper/2506.13009/) (2025)
- [Cascading and Proxy Membership Inference Attacks](https://mlsanity.com/paper/2507.21412/) (2025)
- [How Secure is Forgetting? Linking Machine Unlearning to Machine Learning Attacks](https://mlsanity.com/paper/2503.20257/) (2025)
- [Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection](https://mlsanity.com/paper/2508.15449/) (2025)
- [Understanding the Dilemma of Unlearning for Large Language Models](https://mlsanity.com/paper/2509.24675/) (2025)
- [Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts](https://mlsanity.com/paper/2510.13451/) (2025)
- [Safeguarding Graph Neural Networks against Topology Inference Attacks](https://mlsanity.com/paper/2509.05429/) (2025)
- [Unlearning Information Bottleneck: Machine Unlearning of Systematic Patterns and Biases](https://mlsanity.com/paper/2405.14020/) (2024)
- [OpenLVLM-MIA: A Controlled Benchmark Revealing the Limits of Membership Inference Attacks on Large Vision-Language Models](https://mlsanity.com/paper/2510.16295/) (2025)
- [PrivTune: Efficient and Privacy-Preserving Fine-Tuning of Large Language Models via Device-Cloud Collaboration](https://mlsanity.com/paper/2512.08809/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Nearest Target Is the Wrong One: Target Separation in Arc2Face Identity Unlearning](https://mlsanity.com/paper/2608.30087/) (2026)

## Abstract (arXiv)

Machine unlearning (MU) is essential for enforcing the right to be forgotten in machine learning systems. A key challenge of MU is how to reliably audit whether a model has truly forgotten specified training data. Membership Inference Attacks (MIAs) are widely used for unlearned model auditing, where samples that evade membership detection are regarded as successfully forgotten. We show this assumption is fundamentally flawed: failed membership inference does not imply true forgetting. We prove that unlearned samples occupy fundamentally different positions in the feature space than non-member samples, making this alignment bias unavoidable and unobservable, which leads to systematically optimistic evaluations of unlearning performance. Meanwhile, training shadow models for MIA incurs substantial computational overhead. To address both limitations, we propose Statistical Membership Inference (SMI), a training-free auditing framework that reformulates auditing as estimating the non-member mixture proportion in the unlearned feature distribution. Beyond estimating the forgetting rate, SMI also provides bootstrap reference ranges for quantified auditing reliability. Extensive experiments show that SMI consistently outperforms all MIA-based baselines, with no shadow model training required. Overall, SMI establishes a principled and efficient alternative to MIA-based auditing methods, with both theoretical guarantees and strong empirical performance.