# Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach

arXiv: https://arxiv.org/abs/2102.04655 · HTML: https://mlsanity.com/paper/2102.04655/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Few-Shot Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1905.01723/) (2019)
- [On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization](https://mlsanity.com/paper/1905.03817/) (2019)
- [SGD and Hogwild! Convergence Without the Bounded Gradients Assumption](https://mlsanity.com/paper/1802.03801/) (2018)
- [Good Semi-supervised Learning that Requires a Bad GAN](https://mlsanity.com/paper/1705.09783/) (2017)
- [MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets](https://mlsanity.com/paper/1811.03850/) (2018)
- [Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities](https://mlsanity.com/paper/1701.06264/) (2017)
- [Communication-Efficient Local Decentralized SGD Methods](https://mlsanity.com/paper/1910.09126/) (2019)
- [Learn distributed GAN with Temporary Discriminators](https://mlsanity.com/paper/2007.09221/) (2020)

## Built on by (most cited first, 8 of 8)

- [Better Generative Replay for Continual Federated Learning](https://mlsanity.com/paper/2302.13001/) (2023)
- [Phoenix: A Federated Generative Diffusion Model](https://mlsanity.com/paper/2306.04098/) (2023)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [Federated Learning for Diffusion Models](https://mlsanity.com/paper/2503.06426/) (2025)
- [A Systematic Review of Federated Generative Models](https://mlsanity.com/paper/2405.16682/) (2024)
- [Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memory](https://mlsanity.com/paper/2405.17457/) (2024)
- [PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Models](https://mlsanity.com/paper/2503.08085/) (2025)
- [A Distributed Generative AI Approach for Heterogeneous Multi-Domain Environments under Data Sharing constraints](https://mlsanity.com/paper/2507.12979/) (2025)

## Abstract (arXiv)

Recently, Generative Adversarial Networks (GANs) have demonstrated their potential in federated learning, i.e., learning a centralized model from data privately hosted by multiple sites. A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites. A major theoretical challenge for the federated GAN is the heterogeneity of the local data distributions. Traditional approaches cannot guarantee to learn the target distribution, which isa mixture of the highly different local distributions. This paper tackles this theoretical challenge, and for the first time, provides a provably correct framework for federated GAN. We propose a new approach called Universal Aggregation, which simulates a centralized discriminator via carefully aggregating the mixture of all private discriminators. We prove that a generator trained with this simulated centralized discriminator can learn the desired target distribution. Through synthetic and real datasets, we show that our method can learn the mixture of largely different distributions where existing federated GAN methods fail.