mlsanityData to 7 Oct 2026

PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining

arXiv 2402.09477 ↗First posted 12 Feb 2024cs.CR cs.LGcited by 17

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

We present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMIA eliminates the common dependency of privacy measurement tools on in-distribution non-member data. As a result, PANORAMIA does not modify the model, training data, or training process, and only requires access to a subset of the training data. We evaluate PANORAMIA on ML models for image and tabular data classification, as well as on large-scale language models.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 24 of its 38 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

24 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Bayesian Estimation of Differential Privacy10 Jun
  2. 2021Membership Inference Attacks From First Principles7 Dec

Other references · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2016Wide Residual Networkscited by 3,057
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2016Deep Learning with Differential Privacycited by 2,682
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2015An Introduction to Convolutional Neural Networkscited by 510
  12. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
10 more
  1. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  2. 2020Label-Only Membership Inference Attackscited by 371
  3. 2013The Composition Theorem for Differential Privacycited by 290
  4. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
  5. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  6. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  7. 2023Privacy Auditing with One (1) Training Runcited by 119
  8. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
  9. 2022A General Framework for Auditing Differentially Private Machine Learningcited by 25
  10. 2023Epsilon*: Privacy Metric for Machine Learning Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 5.

17 papers.

Sort:

By area:

  1. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  2. 2025Privacy Auditing of Large Language Modelscited by 22
  3. 2025Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemblecited by 11
  4. 2024Do Parameters Reveal More than Loss for Membership Inference?cited by 9
  5. 2025Unlocking Post-hoc Dataset Inference with Synthetic Datacited by 7
  6. 2025Optimizing Canaries for Privacy Auditing with Metagradient Descentcited by 6
  7. 2024BadGD: A unified data-centric framework to identify gradient descent vulnerabilitiescited by 5
  8. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  9. 2025Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detectioncited by 3
  10. 2025Empirical Privacy Variancecited by 3
  11. 2025In-Context Probing for Membership Inference in Fine-Tuned Language Modelscited by 3
  12. 2026Causal Evaluation of Membership Inference Attackscited by 2
5 more
  1. 2026Impact of Graph Structure on Membership-Inference Risk for Graph Neural Networkscited by 1
  2. 2025Observational Auditing of Label Privacycited by 0
  3. 2026ReproMIA: A Comprehensive Analysis of Model Reprogramming for Proactive Membership Inference Attackscited by 0
  4. 2026On Privacy in Data-Space Tabular Diffusion Models: Influential Factors, Attacker Knowledge, and Metricscited by 0
  5. 2026Natural Identifiers for Privacy and Data Audits in Large Language Modelscited by 0