No Free Lunch for Synthetic Images under Data Scarcity Conditions
On the map: Diffusion models → Robustness, privacy and adversarial aspects
Abstract (arXiv)
This study investigates the trade-offs between fidelity, privacy, and utility in synthetic data generation under conditions of data scarcity and privacy sensitivity. We propose an evaluation framework that jointly assesses these three dimensions and apply it to three widely used generative models, VAE, GAN, and DDPM. The evaluation spans three image datasets, MNIST, OCTMNIST, and OrganAMNIST, encompassing both general-purpose and medical imaging domains. Notable differences arise between the three models in their behaviour when differential privacy mechanisms are introduced during training. GAN and DDPM demonstrate greater robustness, maintaining higher fidelity and downstream utility across a range of noise levels, while VAE degrades more rapidly as privacy constraints increase. This study highlights the importance of a multidimensional evaluation of deep generative models, also noting that their behaviour significantly differs when privacy techniques are applied.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 21 of its 53 references that are in the map. First posted on arXiv 1 Jun 2026.
Builds on
21 of 53 references.
Closest ancestors · 2
Same direction, newest first
- 2024A Gray-box Attack against Latent Diffusion Model-based Image Editing by Posterior Collapse20 Aug
- 2022Differentially Private Diffusion Models18 Oct
Other references · 19
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2022Generative Adversarial Networkscited by 5,603
- 2016Deep Learning with Differential Privacycited by 2,682
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
- 2017Generative Adversarial Networks: An Overviewcited by 588
- 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
- 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 118
- 2021An Introduction to Deep Generative Modelingcited by 104
7 more
- 2022No Free Lunch Theorem for Security and Utility in Federated Learningcited by 39
- 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
- 2024SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Datacited by 27
- 2021Preventing Oversmoothing in VAE via Generalized Variance Parameterizationcited by 15
- 2024Synthetic Tabular Data Validation: A Divergence-Based Approachcited by 6
- 2025Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Frameworkcited by 6
- 2020Differentially Private Variational Autoencoders with Term-wise Gradient Aggregationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Data-Interventional Framework for Auditing Privacy and Fairness in Generative Medical Imaging
- 2025On the MIA Vulnerability Gap Between Private GANs and Diffusion Models
- 2023Differentially Private Diffusion Models Generate Useful Synthetic Images
- 2023PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining
- 2024DP-TLDM: Differentially Private Tabular Latent Diffusion Model
- 2024On Differentially Private 3D Medical Image Synthesis with Controllable Latent Diffusion Models
- 2025Enabling PSO-Secure Synthetic Data Sharing Using Diversity-Aware Diffusion Models
- 2024Leveraging Programmatically Generated Synthetic Data for Differentially Private Diffusion Training