# On the Importance of Conditioning for Privacy-Preserving Data Augmentation

arXiv: https://arxiv.org/abs/2504.05849 · HTML: https://mlsanity.com/paper/2504.05849/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [Deep Learning-based Patient Re-identification Is able to Exploit the Biometric Nature of Medical Chest X-ray Data](https://mlsanity.com/paper/2103.08562/) (2021)
- [Dataset Enhancement with Instance-Level Augmentations](https://mlsanity.com/paper/2406.08249/) (2024)
- [DeepBlur: A Simple and Effective Method for Natural Image Obfuscation](https://mlsanity.com/paper/2104.02655/) (2021)
- [Face De-identification: State-of-the-art Methods and Comparative Studies](https://mlsanity.com/paper/2411.09863/) (2024)
- [P3: Toward Privacy-Preserving Photo Sharing](https://mlsanity.com/paper/1302.5062/) (2013)
- [A False Sense of Privacy: Towards a Reliable Evaluation Methodology for the Anonymization of Biometric Data](https://mlsanity.com/paper/2304.01635/) (2023)

## Abstract (arXiv)

Latent diffusion models can be used as a powerful augmentation method to artificially extend datasets for enhanced training. To the human eye, these augmented images look very different to the originals. Previous work has suggested to use this data augmentation technique for data anonymization. However, we show that latent diffusion models that are conditioned on features like depth maps or edges to guide the diffusion process are not suitable as a privacy preserving method. We use a contrastive learning approach to train a model that can correctly identify people out of a pool of candidates. Moreover, we demonstrate that anonymization using conditioned diffusion models is susceptible to black box attacks. We attribute the success of the described methods to the conditioning of the latent diffusion model in the anonymization process. The diffusion model is instructed to produce similar edges for the anonymized images. Hence, a model can learn to recognize these patterns for identification.