mlsanityData to 7 Oct 2026

Low-Cost High-Power Membership Inference Attacks

arXiv 2312.03262 ↗First posted 6 Dec 2023stat.ML cs.CR cs.LGcited by 99

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership inference attacks aim to detect if a particular data point was used in training a model. We design a novel statistical test to perform robust membership inference attacks (RMIA) with low computational overhead. We achieve this by a fine-grained modeling of the null hypothesis in our likelihood ratio tests, and effectively leveraging both reference models and reference population data samples. RMIA has superior test power compared with prior methods, throughout the TPR-FPR curve (even at extremely low FPR, as low as 0). Under computational constraints, where only a limited number of pre-trained reference models (as few as 1) are available, and also when we vary other elements of the attack (e.g., data distribution), our method performs exceptionally well, unlike prior attacks that approach random guessing. RMIA lays the groundwork for practical yet accurate data privacy risk assessment in machine learning.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 39 of its 61 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

39 of 61 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Scalable Membership Inference Attacks via Quantile Regression7 Jul
  2. 2023Gaussian Membership Inference Privacy12 Jun
  3. 2022Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries19 Oct
  4. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  5. 2022RelaxLoss: Defending Membership Inference Attacks without Losing Utility12 Jul
  6. 2022Bounding Membership Inference24 Feb
  7. 2021Membership Inference Attacks From First Principles7 Dec
  8. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  9. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
  10. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture15 Oct
  11. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models4 Feb

Other references · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
  7. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  8. 2020Label-Only Membership Inference Attackscited by 371
  9. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  10. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  11. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  12. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
16 more
  1. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 183
  2. 2020When Machine Unlearning Jeopardizes Privacycited by 181
  3. 2018Machine Learning with Membership Privacy using Adversarial Regularizationcited by 177
  4. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  5. 2020Membership Leakage in Label-Only Exposurescited by 156
  6. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  7. 2023Privacy Auditing with One (1) Training Runcited by 119
  8. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  9. 2020On the Privacy Risks of Algorithmic Fairnesscited by 82
  10. 2019Differentially Private Model Publishing for Deep Learningcited by 69
  11. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 68
  12. 2015An Exploration of Softmax Alternatives Belonging to the Spherical Loss Familycited by 27
  13. 2022Auditing Membership Leakages of Multi-Exit Networkscited by 26
  14. 2023Leave-one-out Distinguishability in Machine Learningcited by 20
  15. 2020Exploring Alternatives to Softmax Functioncited by 16
  16. 2019An Adaptive and Fast Convergent Approach to Differentially Private Deep Learningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 49; 2026: 33.

99 papers.

Sort:

By area:

  1. 2024Evaluations of Machine Learning Privacy Defenses are Misleadingcited by 35
  2. 2025Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generationcited by 26
  3. 2024Editable Concept Bottleneck Modelscited by 22
  4. 2024Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decodingcited by 20
  5. 2024Context-Aware Membership Inference Attacks against Pre-trained Large Language Modelscited by 20
  6. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  7. 2025Membership Inference Attacks on Large-Scale Models: A Surveycited by 17
  8. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 17
  9. 2025Privacy Attacks on Image AutoRegressive Modelscited by 15
  10. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
  11. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemblecited by 11
  12. 2024Label Smoothing Improves Machine Unlearningcited by 8
38 more
  1. 2025Robust Data Watermarking in Language Models by Injecting Fictitious Knowledgecited by 8
  2. 2025Synthetic Tabular Data: Methods, Attacks and Defensescited by 8
  3. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspectivecited by 8
  4. 2024The Data Minimization Principle in Machine Learningcited by 7
  5. 2024Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learningcited by 6
  6. 2024Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakagecited by 6
  7. 2024Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methodscited by 6
  8. 2025Membership Inference Attacks on Sequence Modelscited by 6
  9. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  10. 2025Train Once, Answer All: Many Pretraining Experiments for the Cost of Onecited by 6
  11. 2024Are Neuromorphic Architectures Inherently Privacy-preserving? An Exploratory Studycited by 5
  12. 2025Practical Bayes-Optimal Membership Inference Attackscited by 5
  13. 2025Privacy Auditing Synthetic Data Release through Local Likelihood Attackscited by 5
  14. 2024On the Impact of Uncertainty and Calibration on Likelihood-Ratio Membership Inference Attackscited by 4
  15. 2024Differentially Private Clustered Federated Learningcited by 4
  16. 2024Defending Membership Inference Attacks via Privacy-aware Sparsity Tuningcited by 4
  17. 2024Provable Privacy Attacks on Trained Shallow Neural Networkscited by 4
  18. 2025Instance-Level Data-Use Auditing of Visual ML Modelscited by 4
  19. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  20. 2025Auditing Approximate Machine Unlearning for Differentially Private Modelscited by 4
  21. 2025SynBench: A Benchmark for Differentially Private Text Generationcited by 4
  22. 2025Synth-MIA: A Testbed for Auditing Privacy Leakage in Tabular Data Synthesiscited by 4
  23. 2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Modelscited by 4
  24. 2024Publishing Neural Networks in Drug Discovery Might Compromise Training Data Privacycited by 3
  25. 2025Membership Inference Risks in Quantized Models: A Theoretical and Empirical Studycited by 3
  26. 2025On the Necessity of Output Distribution Reweighting for Effective Class Unlearningcited by 3
  27. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  28. 2025Reasoning's Razor: Reasoning Improves Accuracy but Can Hurt Recall at Critical Operating Points in Safety and Hallucination Detectioncited by 3
  29. 2026Models Designed to Forget: Machine Unlearning via Key Deletioncited by 3
  30. 2025Hyperparameters in Score-Based Membership Inference Attackscited by 2
  31. 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attackcited by 2
  32. 2025AMUN: Adversarial Machine UNlearningcited by 2
  33. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
  34. 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Reviewcited by 2
  35. 2025Imitative Membership Inference Attackcited by 2
  36. 2026Causal Evaluation of Membership Inference Attackscited by 2
  37. 2025On the Privacy Risks of Spiking Neural Networks: A Membership Inference Analysiscited by 1
  38. 2025Membership Inference Attacks for Unseen Classescited by 1

50 most cited of 99 All 99 (JSON)