# Leveraging Programmatically Generated Synthetic Data for Differentially Private Diffusion Training

arXiv: https://arxiv.org/abs/2412.09842 · HTML: https://mlsanity.com/paper/2412.09842/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 6 of 6)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learning](https://mlsanity.com/paper/2106.04590/) (2021)
- [Private GANs, Revisited](https://mlsanity.com/paper/2302.02936/) (2023)
- [ViP: A Differentially Private Foundation Model for Computer Vision](https://mlsanity.com/paper/2306.08842/) (2023)

## Built on by (most cited first, 1 of 1)

- [Separating Knowledge and Perception with Procedural Data](https://mlsanity.com/paper/2508.11697/) (2025)

## Abstract (arXiv)

Programmatically generated synthetic data has been used in differential private training for classification to enhance performance without privacy leakage. However, as the synthetic data is generated from a random process, the distribution of real data and the synthetic data are distinguishable and difficult to transfer. Therefore, the model trained with the synthetic data generates unrealistic random images, raising challenges to adapt the synthetic data for generative models. In this work, we propose DP-SynGen, which leverages programmatically generated synthetic data in diffusion models to address this challenge. By exploiting the three stages of diffusion models(coarse, context, and cleaning) we identify stages where synthetic data can be effectively utilized. We theoretically and empirically verified that cleaning and coarse stages can be trained without private data, replacing them with synthetic data to reduce the privacy budget. The experimental results show that DP-SynGen improves the quality of generative data by mitigating the negative impact of privacy-induced noise on the generation process.