mlsanityData to 7 Oct 2026

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

arXiv 2606.16952 ↗First posted 15 Jun 2026cs.LG cs.AI stat.APcited by 0

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

The rapid adoption of generative AI and Large Language Models (LLMs) has spurred interest in synthetic data as a privacy-preserving alternative to sensitive real-world datasets. However, generating high-utility synthetic data often carries the risk of memorizing and regurgitating private information from the training corpus. In this work, we present a customizable empirical auditing framework designed to detect and explain such data disclosures. Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures''-where the system incidentally generates a user's data. By partitioning input data into training and holdout sets and applying rigorous statistical hypothesis testing, we determine if observed disclosures are consistent with strict privacy baselines, such as zero-learning or specific Differential Privacy (DP) bounds. Crucially, this approach requires no model access, no canary insertion, and no reference model training -only the synthetic output and a held-out control set. We demonstrate that this framework effectively functions as a membership inference attack, providing empirical lower bounds on privacy leakage that are tighter than prior data-based auditing methods. Our approach is model-agnostic, applies to any synthetic data generation mechanism, and requires orders of magnitude fewer computational resources than shadow-model or canary-based alternatives.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 32 of its 44 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

32 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text19 Feb
  2. 2025Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model8 Feb
  3. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Text4 Mar

Other references · 29

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  4. 2021Membership Inference Attacks From First Principlescited by 839
  5. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  6. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  7. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  8. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  9. 2022Reconstructing Training Data with Informed Adversariescited by 143
  10. 2023Privacy Auditing with One (1) Training Runcited by 119
  11. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  12. 2023Low-Cost High-Power Membership Inference Attackscited by 99
17 more
  1. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  2. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  3. 2023Harnessing large-language models to generate private synthetic textcited by 65
  4. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
  5. 2022Bounding Training Data Reconstruction in Private (Deep) Learningcited by 43
  6. 2024Proving membership in LLM pretraining data via data watermarkscited by 38
  7. 2024Copyright Traps for Large Language Modelscited by 36
  8. 2024Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenoncited by 30
  9. 2023Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishingcited by 28
  10. 2024Private prediction for large-scale synthetic text generationcited by 24
  11. 2018Property Testing for Differential Privacycited by 16
  12. 2025How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacycited by 11
  13. 2025The Surprising Effectiveness of Membership Inference with Simple N-Gram Coveragecited by 7
  14. 2025PANORAMA: A synthetic PII-laced dataset for studying sensitive data memorization in LLMscited by 6
  15. 2025Optimizing Canaries for Privacy Auditing with Metagradient Descentcited by 6
  16. 2024Differentially Private Optimization with Sparse Gradientscited by 2
  17. 2026Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Textcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  2. 2025The DCR Delusion: Measuring the Privacy Risk of Synthetic Data
  3. 2025Does Training with Synthetic Data Truly Protect Privacy?
  4. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  5. 2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesis
  6. 2025PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance
  7. 2021Causally Constrained Data Synthesis for Private Data Release
  8. 2025Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences