mlsanityData to 7 Oct 2026

ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning

arXiv 2007.09339 ↗First posted 18 Jul 2020cs.CR cs.LGcited by 66

Not on the map. Why?

Cited by 34 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

When building machine learning models using sensitive data, organizations should ensure that the data processed in such systems is adequately protected. For projects involving machine learning on personal data, Article 35 of the GDPR mandates it to perform a Data Protection Impact Assessment (DPIA). In addition to the threats of illegitimate access to data through security breaches, machine learning models pose an additional privacy risk to the data by indirectly revealing about it through the model predictions and parameters. Guidances released by the Information Commissioner's Office (UK) and the National Institute of Standards and Technology (US) emphasize on the threat to data from models and recommend organizations to account for and estimate these risks to comply with data protection regulations. Hence, there is an immediate need for a tool that can quantify the privacy risk to data from models. In this paper, we focus on this indirect leakage about training data from machine learning models. We present ML Privacy Meter, a tool that can quantify the privacy risk to data from models through state of the art membership inference attack techniques. We discuss how this tool can help practitioners in compliance with data protection regulations, when deploying machine learning models.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 13; 2024: 13; 2025: 15; 2026: 9.

66 papers.

Sort:

By area:

  1. 2021Membership Inference Attacks From First Principlescited by 839
  2. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  3. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  4. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  5. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  6. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  7. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  8. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 79
  9. 2023Membership Inference of Diffusion Modelscited by 46
  10. 2021Membership Inference on Word Embedding and Beyondcited by 34
  11. 2022Membership Inference Attacks and Generalization: A Causal Perspectivecited by 17
  12. 2024Adversarial Machine Unlearningcited by 16
38 more
  1. 2024De-identification is not enough: a comparison between de-identified and synthetic clinical notescited by 14
  2. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemblecited by 11
  3. 2021SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learningcited by 10
  4. 2024Do Parameters Reveal More than Loss for Membership Inference?cited by 9
  5. 2021Training Data Leakage Analysis in Language Modelscited by 7
  6. 2023Improved Membership Inference Attacks Against Language Classification Modelscited by 7
  7. 2022Data Privacy and Trustworthy Machine Learningcited by 6
  8. 2023Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Developmentcited by 6
  9. 2024Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methodscited by 6
  10. 2025Cascading and Proxy Membership Inference Attackscited by 6
  11. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  12. 2026FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamicscited by 6
  13. 2023Little is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learningcited by 5
  14. 2022Quantitative AI Risk Assessments: Opportunities and Challengescited by 4
  15. 2023FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximationscited by 4
  16. 2023Can Membership Inferencing be Refuted?cited by 4
  17. 2023Fundamental Limits of Membership Inference Attacks on Machine Learning Modelscited by 4
  18. 2024Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvaturecited by 4
  19. 2022On the Discredibility of Membership Inference Attackscited by 3
  20. 2023Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profilecited by 3
  21. 2024Some Targets Are Harder to Identify than Others: Quantifying the Target-dependent Membership Leakagecited by 3
  22. 2024Publishing Neural Networks in Drug Discovery Might Compromise Training Data Privacycited by 3
  23. 2024A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligencecited by 3
  24. 2025Membership Inference Risks in Quantized Models: A Theoretical and Empirical Studycited by 3
  25. 2025FML-bench: Benchmarking Machine Learning Agents for Scientific Researchcited by 3
  26. 2026Revisiting the LiRA Membership Inference Attack Under Realistic Assumptionscited by 3
  27. 2022On the Vulnerability of Data Points under Multiple Membership Inference Attacks and Target Modelscited by 2
  28. 2023Membership Privacy Risks of Sharpness Aware Minimizationcited by 2
  29. 2024Inference Attacks: A Taxonomy, Survey, and Promising Directionscited by 2
  30. 2024Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language modelscited by 2
  31. 2025Imitative Membership Inference Attackcited by 2
  32. 2025Membership Inference over Diffusion-models-based Synthetic Tabular Datacited by 2
  33. 2022Directional Privacy for Deep Learningcited by 1
  34. 2024On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Modelscited by 1
  35. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  36. 2025Spurious Privacy Leakage in Neural Networkscited by 1
  37. 2025Securing AI Systems: A Guide to Known Attacks and Impactscited by 1
  38. 2025Amulet: a Python Library for Assessing Interactions Among ML Defenses and Riskscited by 1

50 most cited of 66 All 66 (JSON)