Learning-Based Difficulty Calibration for Enhanced Membership Inference Attacks
On the map: LLM safety and security → Membership inference attacks
Abstract (arXiv)
Machine learning models, in particular deep neural networks, are currently an integral part of various applications, from healthcare to finance. However, using sensitive data to train these models raises concerns about privacy and security. One method that has emerged to verify if the trained models are privacy-preserving is Membership Inference Attacks (MIA), which allows adversaries to determine whether a specific data point was part of a model's training dataset. While a series of MIAs have been proposed in the literature, only a few can achieve high True Positive Rates (TPR) in the low False Positive Rate (FPR) region (0.01%~1%). This is a crucial factor to consider for an MIA to be practically useful in real-world settings. In this paper, we present a novel approach to MIA that is aimed at significantly improving TPR at low FPRs. Our method, named learning-based difficulty calibration for MIA(LDC-MIA), characterizes data records by their hardness levels using a neural network classifier to determine membership. The experiment results show that LDC-MIA can improve TPR at low FPR by up to 4x compared to the other difficulty calibration based MIAs. It also has the highest Area Under ROC curve (AUC) across all datasets. Our method's cost is comparable with most of the existing MIAs, but is orders of magnitude more efficient than one of the state-of-the-art methods, LiRA, while achieving similar performance.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 24 of its 43 references that are in the map. First posted on arXiv 10 Jan 2024.
Builds on
24 of 43 references.
Closest ancestors · 6
Same direction, newest first
- 2023Students Parrot Their Teachers: Membership Inference on Model Distillation6 Mar
- 2022Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Models22 Aug
- 2021Membership Inference Attacks From First Principles7 Dec
- 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
- 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture15 Oct
- 2021Practical Blind Membership Inference Attack via Differential Comparisons5 Jan
Other references · 18
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2016Wide Residual Networkscited by 3,057
- 2016Deep Learning with Differential Privacycited by 2,682
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
- 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
- 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
- 2020Label-Only Membership Inference Attackscited by 371
- 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
- 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
6 more
- 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
- 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
- 2015Brain Tumor Segmentation with Deep Neural Networkscited by 137
- 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
- 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 83
- 2022Auditing Membership Leakages of Multi-Exit Networkscited by 26
Built on by
Citing papers per year: 2026: 2.
2 papers.