mlsanityData to 13 Oct 2026

FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models

arXiv 2407.14730 ↗First posted 20 Jul 2024cs.LG cs.CV cs.DCcited by 5

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

We introduce FedDM, a novel training framework designed for the federated training of diffusion models. Our theoretical analysis establishes the convergence of diffusion models when trained in a federated setting, presenting the specific conditions under which this convergence is guaranteed. We propose a suite of training algorithms that leverage the U-Net architecture as the backbone for our diffusion models. These include a basic Federated Averaging variant, FedDM-vanilla, FedDM-prox to handle data heterogeneity among clients, and FedDM-quant, which incorporates a quantization module to reduce the model update size, thereby enhancing communication efficiency across the federated network. We evaluate our algorithms on FashionMNIST (28x28 resolution), CIFAR-10 (32x32 resolution), and CelebA (64x64 resolution) for DDPMs, as well as LSUN Church Outdoors (256x256 resolution) for LDMs, focusing exclusively on the imaging modality. Our evaluation results demonstrate that FedDM algorithms maintain high generation quality across image resolutions. At the same time, the use of quantized updates and proximal terms in the local training objective significantly enhances communication efficiency (up to 4x) and model convergence, particularly in non-IID data settings, at the cost of increased FID scores (up to 1.75x).

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 48 of its 61 references that are in the map. First posted on arXiv 20 Jul 2024.

Builds on

48 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Training Diffusion Models with Federated Learning18 Jun
  2. 2023Phoenix: A Federated Generative Diffusion Model7 Jun
  3. 2023Ablating Concepts in Text-to-Image Diffusion Models23 Mar
  4. 2023Extracting Training Data from Diffusion Models30 Jan

Other references · 44

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  3. 2020Denoising Diffusion Implicit Modelscited by 9,718
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  7. 2022Generative Adversarial Networkscited by 5,614
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,484
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,077
  11. 2013Auto-Encoding Variational Bayescited by 4,024
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,751
32 more
  1. 2014Deep Learning Face Attributes in the Wildcited by 3,686
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  3. 2022Video Diffusion Modelscited by 2,293
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,298
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 846
  7. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 496
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  9. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shiftingcited by 364
  10. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  11. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 350
  12. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  13. 2022Diffusion Models already have a Semantic Latent Spacecited by 325
  14. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  15. 2022Post-training Quantization on Diffusion Modelscited by 267
  16. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 254
  17. 2022Diffusion Models for Medical Anomaly Detectioncited by 232
  18. 2023TextDiffuser: Diffusion Models as Text Painterscited by 224
  19. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  20. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 133
  21. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
  22. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 63
  23. 2022MDM: Molecular Diffusion Model for 3D Molecule Generationcited by 58
  24. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
  25. 2021DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learningcited by 41
  26. 2023FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clientscited by 23
  27. 2022Wireless Quantized Federated Learning: A Joint Computation and Communication Designcited by 15
  28. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  29. 2019Balancing Reconstruction Quality and Regularisation in ELBO for VAEscited by 9
  30. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Modelscited by 5
  31. 2024CollaFuse: Collaborative Diffusion Modelscited by 4
  32. 2024Expediting In-Network Federated Learning by Voting-Based Consensus Model Compressioncited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025FedSDA: Federated Stain Distribution Alignment for Non-IID Histopathological Image Classificationcited by 2
  2. 2025Personalized Federated Training of Diffusion Models with Privacy Guaranteescited by 0
  3. 2025Federated Flow Matchingcited by 0
  4. 2026Compositional Generative Modeling from Decentralized Datacited by 0
  5. 2026Generative AI and Federated Learning for Intrusion Detection Systems: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models
  2. 2025Federated Learning for Diffusion Models
  3. 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning
  4. 2026Heterogeneous Decentralized Diffusion Models
  5. 2024FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models
  6. 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
  7. 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Model
  8. 2023One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models