mlsanityData to 7 Oct 2026

Does Training with Synthetic Data Truly Protect Privacy?

arXiv 2502.12976 ↗First posted 18 Feb 2025cs.CR cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

As synthetic data becomes increasingly popular in machine learning tasks, numerous methods--without formal differential privacy guarantees--use synthetic data for training. These methods often claim, either explicitly or implicitly, to protect the privacy of the original training data. In this work, we explore four different training paradigms: coreset selection, dataset distillation, data-free knowledge distillation, and synthetic data generated from diffusion models. While all these methods utilize synthetic data for training, they lead to vastly different conclusions regarding privacy preservation. We caution that empirical approaches to preserving data privacy require careful and rigorous evaluation; otherwise, they risk providing a false sense of privacy.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

39 of 46 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  6. 2018Group Normalizationcited by 1,514
  7. 2017Continual Learning with Deep Generative Replaycited by 1,152
  8. 2021Membership Inference Attacks From First Principlescited by 839
  9. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  10. 2020Dataset Condensation with Gradient Matchingcited by 510
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  12. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
27 more
  1. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  3. 2021Dataset Condensation with Distribution Matchingcited by 362
  4. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  5. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  6. 2022Scaling Up Dataset Distillation to ImageNet-1K with Constant Memorycited by 180
  7. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 175
  8. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
  9. 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
  10. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 119
  11. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  12. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 64
  13. 2022Accelerating Dataset Distillation via Model Augmentationcited by 62
  14. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  15. 2020Introduction to Core-sets: an Updated Surveycited by 45
  16. 2021The 2021 Image Similarity Dataset and Challengecited by 44
  17. 2022RelaxLoss: Defending Membership Inference Attacks without Losing Utilitycited by 35
  18. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  19. 2024Evaluations of Machine Learning Privacy Defenses are Misleadingcited by 35
  20. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  21. 2023Overconfidence is a Dangerous Thing: Mitigating Membership Inference Attacks by Enforcing Less Confident Predictioncited by 26
  22. 2022No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"cited by 23
  23. 2023Federated Generative Learning with Foundation Modelscited by 23
  24. 2024DiffClass: Diffusion-Based Class Incremental Learningcited by 23
  25. 2022IDEAL: Query-Efficient Data-Free Learning from Black-box Modelscited by 16
  26. 2023A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairnesscited by 9
  27. 2023Sampling to Distill: Knowledge Transfer from Open-World Datacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025AI Agent Behavioral Sciencecited by 5
  2. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
  3. 2025LCMem: A Universal Model for Robust Image Memorization Detectioncited by 4
  4. 2025CompLeak: Deep Learning Model Compression Exacerbates Privacy Leakagecited by 3
  5. 2026CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Trainingcited by 2
  6. 2026Decoupling Generalizability and Membership Privacy Risks in Neural Networkscited by 1
  7. 2026RedacBench: Can AI Erase Your Secrets?cited by 1
  8. 2026Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implicationscited by 1
  9. 2026Studying the Role of Synthetic Data for Machine Learning-based Wireless Networks Traffic Forecastingcited by 0
  10. 2026Privacy Amplification Persists under Unlimited Synthetic Data Releasecited by 0
  11. 2026Learnability and Privacy Vulnerability are Entangled in a Few Critical Weightscited by 0
  12. 2026Causal-Privacy Audit Workflow for Synthetic and Distilled Data in Dropout Supportcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  2. 2024Leveraging Programmatically Generated Synthetic Data for Differentially Private Diffusion Training
  3. 2023Differentially Private Diffusion Models Generate Useful Synthetic Images
  4. 2021Causally Constrained Data Synthesis for Private Data Release
  5. 2023Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models
  6. 2026Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data
  7. 2023On the Inherent Privacy Properties of Discrete Denoising Diffusion Models
  8. 2025Personalized Federated Training of Diffusion Models with Privacy Guarantees