mlsanityData to 7 Oct 2026

SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning

arXiv 2212.10986 ↗First posted 21 Dec 2022cs.LG cs.CR cs.GTcited by 47

Not on the map. Why?

Abstract (arXiv)

Deploying machine learning models in production may allow adversaries to infer sensitive information about training data. There is a vast literature analyzing different types of inference risks, ranging from membership inference to reconstruction attacks. Inspired by the success of games (i.e., probabilistic experiments) to study security properties in cryptography, some authors describe privacy inference risks in machine learning using a similar game-based style. However, adversary capabilities and goals are often stated in subtly different ways from one presentation to the other, which makes it hard to relate and compose results. In this paper, we present a game-based framework to systematize the body of knowledge on privacy inference risks in machine learning. We use this framework to (1) provide a unifying structure for definitions of inference risks, (2) formally establish known relations among definitions, and (3) to uncover hitherto unknown relations that would have been difficult to spot otherwise.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 53 of its 89 references that are in the map. First posted on arXiv 21 Dec 2022.

Builds on

53 of 89 references.

References · 53

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  4. 2021Membership Inference Attacks From First Principlescited by 839
  5. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  6. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  7. 2020Label-Only Membership Inference Attackscited by 371
  8. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 285
  9. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  10. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  11. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  12. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
38 more
  1. 2020Membership Leakage in Label-Only Exposurescited by 156
  2. 2020A Survey of Privacy Attacks in Machine Learningcited by 143
  3. 2022Reconstructing Training Data with Informed Adversariescited by 143
  4. 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 128
  5. 2019Reconstruction and Membership Inference Attacks against Generative Modelscited by 122
  6. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 117
  7. 2018Stealing Hyperparameters in Machine Learningcited by 115
  8. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  9. 2020Synthetic Data -- Anonymisation Groundhog Daycited by 105
  10. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  11. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 89
  12. 2020On the Privacy Risks of Algorithmic Fairnesscited by 82
  13. 2021Practical Blind Membership Inference Attack via Differential Comparisonscited by 73
  14. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  15. 2020When Machine Learning Meets Privacy: A Survey and Outlookcited by 67
  16. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  17. 2021Property Inference From Poisoningcited by 56
  18. 2021Membership Inference Attacks Against Recommender Systemscited by 52
  19. 2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 50
  20. 2021Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implicationscited by 50
  21. 2020Improving Robustness to Model Inversion Attacks via Mutual Information Regularizationcited by 49
  22. 2022Are Attribute Inference Attacks Just Imputation?cited by 48
  23. 2020Leakage of Dataset Properties in Multi-Party Machine Learningcited by 46
  24. 2021Formalizing and Estimating Distribution Inference Riskscited by 42
  25. 2022Debugging Differential Privacy: A Case Study for Privacy Auditingcited by 39
  26. 2021Membership Inference on Word Embedding and Beyondcited by 34
  27. 2019Disparate Vulnerability to Membership Inference Attackscited by 33
  28. 2019SoK: Differential Privaciescited by 33
  29. 2021Property Inference Attacks Against GANscited by 32
  30. 2022Bayesian Estimation of Differential Privacycited by 31
  31. 2019Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentationcited by 29
  32. 2022SNAP: Efficient Extraction of Private Properties with Poisoningcited by 29
  33. 2017LOGAN: Membership Inference Attacks Against Generative Modelscited by 28
  34. 2020Investigating Membership Inference Attacks under Data Dependenciescited by 27
  35. 2019Towards Formalizing the GDPR's Notion of Singling Outcited by 25
  36. 2022Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learningcited by 19
  37. 2022Subject Membership Inference Attacks in Federated Learningcited by 18
  38. 2022Distribution inference risks: Identifying and mitigating sources of leakagecited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 13; 2025: 13; 2026: 13.

47 papers.

Sort:

By area:

  1. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  2. 2023User Inference Attacks on Large Language Modelscited by 36
  3. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  4. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  5. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  6. 2023Reinforcement Unlearningcited by 16
  7. 2024Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attackscited by 13
  8. 2025The DCR Delusion: Measuring the Privacy Risk of Synthetic Datacited by 11
  9. 2025Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacycited by 11
  10. 2024The Mosaic Memory of Large Language Modelscited by 9
  11. 2024Do Parameters Reveal More than Loss for Membership Inference?cited by 9
  12. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspectivecited by 8
35 more
  1. 2023SoK: Unintended Interactions among Machine Learning Defenses and Riskscited by 6
  2. 2025Gaussian DP for Reporting Differential Privacy Guarantees in Machine Learningcited by 6
  3. 2024State-of-the-Art Approaches to Enhancing Privacy Preservation of Machine Learning Datasets: A Surveycited by 5
  4. 2025VISTA: Vision-Language Inference for Training-Free Stock Time-Series Analysiscited by 5
  5. 2025SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmarkcited by 5
  6. 2025How do we measure privacy in text? A survey of text anonymization metricscited by 5
  7. 2024Analyzing Inference Privacy Risks Through Gradients in Machine Learningcited by 4
  8. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  9. 2024Publishing Neural Networks in Drug Discovery Might Compromise Training Data Privacycited by 3
  10. 2024Learning Robust and Privacy-Preserving Representations via Information Theorycited by 3
  11. 2023Why Train More? Effective and Efficient Membership Inference via Memorizationcited by 2
  12. 2023From Principle to Practice: Vertical Data Minimization for Machine Learningcited by 2
  13. 2024SoK: Security and Privacy Risks of Healthcare AIcited by 2
  14. 2026Causal Evaluation of Membership Inference Attackscited by 2
  15. 2024Ensembler: Protect Collaborative Inference Privacy from Model Inversion Attack via Selective Ensemblecited by 1
  16. 2026TOSSS: a CVE-based Software Security Benchmark for Large Language Modelscited by 1
  17. 2026Beyond Indistinguishability: Measuring Extraction Risk in LLM APIscited by 1
  18. 2026On Reliability of Membership Inference Vulnerability Evaluationcited by 1
  19. 2026Probing Memorization of Tabular In-Context Learningcited by 1
  20. 2023Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomycited by 0
  21. 2024Privacy and Security Implications of Cloud-Based AI Services : A Surveycited by 0
  22. 2024Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Modelscited by 0
  23. 2024Calibrating Practical Privacy Risks for Differentially Private Machine Learningcited by 0
  24. 2025Counterfactual Influence as a Distributional Quantitycited by 0
  25. 2025Membership Inference Attack with Partial Featurescited by 0
  26. 2025Privacy Risks in Time Series Forecasting: User- and Record-Level Membership Inferencecited by 0
  27. 2025Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacycited by 0
  28. 2026SoK: Enhancing Cryptographic Collaborative Learning with Differential Privacycited by 0
  29. 2026On Privacy in Data-Space Tabular Diffusion Models: Influential Factors, Attacker Knowledge, and Metricscited by 0
  30. 2026AI Security Research Should Better Incentivize Defense Researchcited by 0
  31. 2026SoK: Colluding Adversaries in Machine Learning Pipelinescited by 0
  32. 2026Characterizing Privacy-Audit Alignment in Behavioral Audit of Machine Unlearningcited by 0
  33. 2026Position Matters: Feature Inversion Attacks in ViT Split Inference with Token Reduction and Shufflingcited by 0
  34. 2026Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Modelscited by 0
  35. 2026Conformal Prediction for Offensive Securitycited by 0