mlsanityData to 14 Oct 2026

Membership Inference Attacks with False Discovery Rate Control

arXiv 2508.07066 ↗First posted 9 Aug 2025stat.ML cs.CV cs.LGcited by 2

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Recent studies have shown that deep learning models are vulnerable to membership inference attacks (MIAs), which aim to infer whether a data record was used to train a target model or not. To analyze and study these vulnerabilities, various MIA methods have been proposed. Despite the significance and popularity of MIAs, existing works on MIAs are limited in providing guarantees on the false discovery rate (FDR), which refers to the expected proportion of false discoveries among the identified positive discoveries. However, it is very challenging to ensure the false discovery rate guarantees, because the underlying distribution is usually unknown, and the estimated non-member probabilities often exhibit interdependence. To tackle the above challenges, in this paper, we design a novel membership inference attack method, which can provide the guarantees on the false discovery rate. Additionally, we show that our method can also provide the marginal probability guarantee on labeling true non-member data as member data. Notably, our method can work as a wrapper that can be seamlessly integrated with existing MIA methods in a post-hoc manner, while also providing the FDR control. We perform the theoretical analysis for our method. Extensive experiments in various settings (e.g., the black-box setting and the lifelong learning setting) are also conducted to verify the desirable performance of our method.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 59 of its 85 references that are in the map. First posted on arXiv 9 Aug 2025.

Builds on

59 of 85 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Order of Magnitude Speedups for LLM Membership Inference22 Sep
  2. 2024Shadow-Free Membership Inference Attacks: Recommender Systems Are More Vulnerable Than You Thought11 May
  3. 2023Generated Distributions Are All You Need for Membership Inference Attacks Against Generative Models30 Oct
  4. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study29 Sep
  5. 2023Scalable Membership Inference Attacks via Quantile Regression7 Jul
  6. 2023Membership Inference Attacks against Diffusion Models7 Feb
  7. 2022Semi-Leak: Membership Inference Attacks Against Semi-supervised Learning25 Jul
  8. 2022Membership Inference Attack Using Self Influence Functions26 May
  9. 2021Membership Inference Attacks From First Principles7 Dec
  10. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  11. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
  12. 2021Membership Inference Attacks Against Recommender Systems16 Sep
2 more
  1. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar
  2. 2021Membership Inference Attacks are Easier on Difficult Problems15 Feb

Other references · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2023Segment Anythingcited by 8,234
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  8. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  10. 2019Machine Unlearningcited by 1,061
  11. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  12. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
33 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  2. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  3. 2019Certified Data Removal from Machine Learning Modelscited by 535
  4. 2019Conformal Prediction Under Covariate Shiftcited by 519
  5. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  6. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  7. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  8. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  9. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 279
  10. 2021Machine Unlearning of Features and Labelscited by 229
  11. 2020When Machine Unlearning Jeopardizes Privacycited by 198
  12. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  13. 2023Putting the Object Back into Video Object Segmentationcited by 188
  14. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  15. 2021Testing for Outliers with Conformal p-valuescited by 134
  16. 2023Boundary Unlearningcited by 134
  17. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 116
  18. 2014Controlling the false discovery rate via knockoffscited by 97
  19. 2020Estimating Example Difficulty Using Variance of Gradientscited by 97
  20. 2020Subpopulation Data Poisoning Attackscited by 70
  21. 2022Practical Adversarial Multivalid Conformal Predictioncited by 65
  22. 2019Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentationcited by 32
  23. 2022Blur Interpolation Transformer for Real-World Motion from Blurcited by 25
  24. 2021Membership Inference Attacks on Deep Regression Models for Neuroimagingcited by 22
  25. 2022Leveraging Adversarial Examples to Quantify Membership Information Leakagecited by 19
  26. 2022Integrative conformal p-values for powerful out-of-distribution testing with labeled outlierscited by 19
  27. 2024Robust Yet Efficient Conformal Prediction Setscited by 16
  28. 2020Conditional calibration for false discovery rate control under dependencecited by 11
  29. 2024Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learningcited by 11
  30. 2018Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Modelscited by 10
  31. 2024Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Predictioncited by 10
  32. 2023Masked Autoencoders are Scalable Learners of Cellular Morphologycited by 9
  33. 2018False Discovery Rate Control via Debiased Lassocited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Quantifying Uncertainty in Natural Language Explanations of Large Language Models for Question Answeringcited by 3
  2. 2026Uncertainty-aware Language Guidance for Concept Bottleneck Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021To Trust or Not To Trust Prediction Scores for Membership Inference Attacks
  2. 2024Learning-Based Difficulty Calibration for Enhanced Membership Inference Attacks
  3. 2022MIAShield: Defending Membership Inference Attacks via Preemptive Exclusion of Members
  4. 2026On Reliability of Membership Inference Vulnerability Evaluation
  5. 2025Membership Inference Attacks Beyond Overfitting
  6. 2023Membership Inference Attacks on DNNs using Adversarial Perturbations
  7. 2022Membership-Doctor: Comprehensive Assessment of Membership Inference Against Machine Learning Models
  8. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble