mlsanityData to 7 Oct 2026

CipherDM: Secure Three-Party Inference for Diffusion Model Sampling

arXiv 2409.05414 ↗First posted 9 Sep 2024cs.CR cs.AI cs.CVcited by 3

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Diffusion Models (DMs) achieve state-of-the-art synthesis results in image generation and have been applied to various fields. However, DMs sometimes seriously violate user privacy during usage, making the protection of privacy an urgent issue. Using traditional privacy computing schemes like Secure Multi-Party Computation (MPC) directly in DMs faces significant computation and communication challenges. To address these issues, we propose CipherDM, the first novel, versatile and universal framework applying MPC technology to DMs for secure sampling, which can be widely implemented on multiple DM based tasks. We thoroughly analyze sampling latency breakdown, find time-consuming parts and design corresponding secure MPC protocols for computing nonlinear activations including SoftMax, SiLU and Mish. CipherDM is evaluated on popular architectures (DDPM, DDIM) using MNIST dataset and on SD deployed by diffusers. Compared to direct implementation on SPU, our approach improves running time by approximately 1.084× ~ 2.328×, and reduces communication costs by approximately 1.212× ~ 1.791×.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 35 of its 56 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

35 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Diffence: Fencing Membership Privacy With Diffusion Models7 Dec
  2. 2022Differentially Private Diffusion Models18 Oct

Other references · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
21 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  6. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  7. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  8. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  9. 2022Diffusion Models for Medical Anomaly Detectioncited by 231
  10. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  11. 2020The Discrete Gaussian for Differential Privacycited by 134
  12. 2020CrypTFlow2: Practical 2-Party Secure Inferencecited by 82
  13. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
  14. 2019CrypTFlow: Secure TensorFlow Inferencecited by 61
  15. 2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
  16. 2023PUMA: Secure Inference of LLaMA-7B in Five Minutescited by 44
  17. 2021MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understandingcited by 36
  18. 2023Diff-Privacy: Diffusion-based Face Privacy Protectioncited by 25
  19. 2020Near Instance-Optimality in Differential Privacycited by 23
  20. 2022MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attentioncited by 19
  21. 2023EasyPhoto: Your Smart AI Photo Generatorcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Buster: Implanting Semantic Backdoor into Text Encoder to Mitigate NSFW Content Generationcited by 3
  2. 2024Antelope: Potent and Concealed Jailbreak Attack Strategycited by 2
  3. 2026Public Diffusion Models, Private Images: Key-Controlled Inversion for Conditional Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Preserving Diffusion Model Using Homomorphic Encryption
  2. 2024Dual-Model Defense: Safeguarding Diffusion Models from Membership Inference Attacks through Disjoint Data Splitting
  3. 2024Gradient Inversion of Federated Diffusion Models
  4. 2023Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy
  5. 2024Is Diffusion Model Safe? Severe Data Leakage via Gradient-Guided Diffusion Model
  6. 2024Differentially Private Fine-Tuning of Diffusion Models
  7. 2023Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation
  8. 2025Enhanced Privacy Leakage from Noise-Perturbed Gradients via Gradient-Guided Conditional Diffusion Models