# Privacy Distillation: Reducing Re-identification Risk of Multimodal Diffusion Models

arXiv: https://arxiv.org/abs/2306.01322 · HTML: https://mlsanity.com/paper/2306.01322/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 40 of its 50 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning](https://mlsanity.com/paper/1702.07464/) (2017)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [RoentGen: Vision-Language Foundation Model for Chest X-ray Generation](https://mlsanity.com/paper/2211.12737/) (2022)
- [Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification](https://mlsanity.com/paper/2012.03173/) (2020)
- [Improved Techniques for Training Single-Image GANs](https://mlsanity.com/paper/2003.11512/) (2020)

## Built on by (most cited first, 18 of 18)

- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [White-box Membership Inference Attacks against Diffusion Models](https://mlsanity.com/paper/2308.06405/) (2023)
- [Black-box Membership Inference Attacks against Fine-tuned Diffusion Models](https://mlsanity.com/paper/2312.08207/) (2023)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [RadEdit: stress-testing biomedical vision models via diffusion image editing](https://mlsanity.com/paper/2312.12865/) (2023)
- [From Pixel to Cancer: Cellular Automata in Computed Tomography](https://mlsanity.com/paper/2403.06459/) (2024)
- [On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook](https://mlsanity.com/paper/2307.16680/) (2023)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Unconditional Latent Diffusion Models Memorize Patient Imaging Data: Implications for Openly Sharing Synthetic Data](https://mlsanity.com/paper/2402.01054/) (2024)
- [One-Step Offline Distillation of Diffusion-based Models via Koopman Modeling](https://mlsanity.com/paper/2505.13358/) (2025)
- [CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs](https://mlsanity.com/paper/2505.10496/) (2025)
- [MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection](https://mlsanity.com/paper/2405.19458/) (2024)
- [The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generation](https://mlsanity.com/paper/2502.07516/) (2025)
- [SoK: Memorisation in machine learning](https://mlsanity.com/paper/2311.03075/) (2023)
- [SoK: Can Synthetic Images Replace Real Data? A Survey of Utility and Privacy of Synthetic Image Generation](https://mlsanity.com/paper/2506.19360/) (2025)
- [RELICT: A Replica Detection Framework for Medical Image Generation](https://mlsanity.com/paper/2502.17360/) (2025)
- [MedForge: Interpretable Medical Deepfake Detection via Forgery-aware Reasoning](https://mlsanity.com/paper/2603.18577/) (2026)
- [Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks](https://mlsanity.com/paper/2609.10608/) (2026)

## Abstract (arXiv)

Knowledge distillation in neural networks refers to compressing a large model or dataset into a smaller version of itself. We introduce Privacy Distillation, a framework that allows a text-to-image generative model to teach another model without exposing it to identifiable data. Here, we are interested in the privacy issue faced by a data provider who wishes to share their data via a multimodal generative model. A question that immediately arises is ``How can a data provider ensure that the generative model is not leaking identifiable information about a patient?''. Our solution consists of (1) training a first diffusion model on real data (2) generating a synthetic dataset using this model and filtering it to exclude images with a re-identifiability risk (3) training a second diffusion model on the filtered synthetic data only. We showcase that datasets sampled from models trained with privacy distillation can effectively reduce re-identification risk whilst maintaining downstream performance.