mlsanityData to 7 Oct 2026

Training Diffusion Models with Federated Learning

arXiv 2406.12575 ↗First posted 18 Jun 2024cs.LG cs.DCcited by 15

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

The training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies, raising concerns about privacy, copyright, and data authority due to their lack of transparency regarding training data. To ad-dress this issue, we propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data. Our approach adapts the Federated Averaging (FedAvg) algorithm to train a Denoising Diffusion Model (DDPM). Through a novel utilization of the underlying UNet backbone, we achieve a significant reduction of up to 74% in the number of parameters exchanged during training,compared to the naive FedAvg approach, whilst simultaneously maintaining image quality comparable to the centralized setting, as evaluated by the FID score.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 33 of its 43 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

33 of 43 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
21 more
  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  10. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  11. 2022Diffusion Models in Vision: A Surveycited by 1,008
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  13. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  14. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  15. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  16. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 89
  17. 2021Copyright in Generative Deep Learningcited by 29
  18. 2020Federated Generative Adversarial Learningcited by 19
  19. 2022Correlated quantization for distributed mean estimation and optimizationcited by 14
  20. 2022Aergia: Leveraging Heterogeneity in Federated Learning Systemscited by 6
  21. 2022FedUKD: Federated UNet Model with Knowledge Distillation for Land Use Classification from Satellite and Street Viewscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 4.

15 papers.

Sort:

By area:

  1. 2024Deep Generative Models for 3D Medical Image Synthesiscited by 14
  2. 2025Federated Learning for Diffusion Modelscited by 6
  3. 2024FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Modelscited by 5
  4. 2024CollaFuse: Collaborative Diffusion Modelscited by 4
  5. 2024Federated Diffusion Modeling with Differential Privacy for Tabular Data Synthesiscited by 2
  6. 2025FedCGD: Collective Gradient Divergence Optimized Scheduling for Wireless Federated Learningcited by 2
  7. 2025FedSDA: Federated Stain Distribution Alignment for Non-IID Histopathological Image Classificationcited by 2
  8. 2025Fed-NDIF: A Noise-Embedded Federated Diffusion Model For Low-Count Whole-Body PET Denoisingcited by 1
  9. 2025FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Modelscited by 1
  10. 2026On the Tradeoffs of On-Device Generative Models in Federated Predictive Maintenance Systemscited by 1
  11. 2025Personalized Federated Training of Diffusion Models with Privacy Guaranteescited by 0
  12. 2025Federated Flow Matchingcited by 0
3 more
  1. 2026Dynamic Topology Optimization for Non-IID Data in Decentralized Learningcited by 0
  2. 2026UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignmentcited by 0
  3. 2026Compositional Generative Modeling from Decentralized Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Phoenix: A Federated Generative Diffusion Model
  2. 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning
  3. 2023Federated Learning with Diffusion Models for Privacy-Sensitive Vision Tasks
  4. 2026Heterogeneous Decentralized Diffusion Models
  5. 2023Masked Diffusion Models Are Fast Distribution Learners
  6. 2024pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning
  7. 2023Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models
  8. 2025Paris: A Decentralized Trained Open-Weight Diffusion Model