mlsanityData to 7 Oct 2026

A Critical Review on the Effectiveness and Privacy Threats of Membership Inference Attacks

arXiv 2603.22987 ↗First posted 24 Mar 2026cs.CR cs.LGcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Membership inference attacks (MIAs) aim to determine whether a data sample was included in a machine learning (ML) model's training set and have become the de facto standard for measuring privacy leakages in ML. We propose an evaluation framework that defines the conditions under which MIAs constitute a genuine privacy threat, and review representative MIAs against it. We find that, under the realistic conditions defined in our framework, MIAs represent weak privacy threats. Thus, relying on them as a privacy metric in ML can lead to an overestimation of risk and to unnecessary sacrifices in model utility as a consequence of employing too strong defenses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 41 of its 79 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

41 of 79 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions8 Mar
  2. 2024Is Difficulty Calibration All We Need? Towards More Practical Membership Inference Attacks31 Aug
  3. 2024Do Parameters Reveal More than Loss for Membership Inference?17 Jun
  4. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  5. 2023Low-Cost High-Power Membership Inference Attacks6 Dec
  6. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  7. 2023MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training2 Nov
  8. 2023Fundamental Limits of Membership Inference Attacks on Machine Learning Models20 Oct
  9. 2023Scalable Membership Inference Attacks via Quantile Regression7 Jul
  10. 2022On the Discredibility of Membership Inference Attacks6 Dec
  11. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  12. 2021Membership Inference Attacks From First Principles7 Dec
3 more
  1. 2021Enhanced Membership Inference Attacks against Machine Learning Models18 Nov
  2. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov
  3. 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar

Other references · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2017Random Erasing Data Augmentationcited by 1,235
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  10. 2020Label-Only Membership Inference Attackscited by 371
  11. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  12. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
14 more
  1. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  2. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  3. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 83
  4. 2021BertGCN: Transductive Text Classification by Combining GCN and BERTcited by 74
  5. 2020ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learningcited by 66
  6. 2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
  7. 2020No Routing Needed Between Capsulescited by 27
  8. 2020Investigating Membership Inference Attacks under Data Dependenciescited by 27
  9. 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
  10. 2016Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddingscited by 11
  11. 2022Efficient Adaptive Ensembling for Image Classificationcited by 8
  12. 2025A Consensus Privacy Metrics Framework for Synthetic Datacited by 8
  13. 2018A Statistical Approach to Adult Census Income Level Predictioncited by 7
  14. 2024SATA: Spatial Autocorrelation Token Analysis for Enhancing the Robustness of Vision Transformerscited by 2

Built on by

Not cited yet.