# Decentralized Learning of Generative Adversarial Networks from Non-iid Data

arXiv: https://arxiv.org/abs/1905.09684 · HTML: https://mlsanity.com/paper/1905.09684/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 10 of 10)

- [FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks](https://mlsanity.com/paper/2201.03169/) (2022)
- [Consistent Distributed Ranking of Generative Models via Kernel Distances](https://mlsanity.com/paper/2310.11714/) (2023)
- [Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent](https://mlsanity.com/paper/2104.08100/) (2021)
- [EFFGAN: Ensembles of fine-tuned federated GANs](https://mlsanity.com/paper/2206.11682/) (2022)
- [PaDPaF: Partial Disentanglement with Partially-Federated GANs](https://mlsanity.com/paper/2212.03836/) (2022)
- [Continual Learning of Generative Models with Limited Data: From Wasserstein-1 Barycenter to Adaptive Coalescence](https://mlsanity.com/paper/2101.09225/) (2021)
- [Modeling Global Distribution for Federated Learning with Label Distribution Skew](https://mlsanity.com/paper/2212.08883/) (2022)
- [A New Distributed Method for Training Generative Adversarial Networks](https://mlsanity.com/paper/2107.08681/) (2021)
- [FissionVAE: Federated Non-IID Image Generation with Latent Space and Decoder Decomposition](https://mlsanity.com/paper/2408.17090/) (2024)
- [FedCAR: Cross-client Adaptive Re-weighting for Generative Models in Federated Learning](https://mlsanity.com/paper/2412.11463/) (2024)

## Abstract (arXiv)

This work addresses a new problem that learns generative adversarial networks (GANs) from multiple data collections that are each i) owned separately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such non-iid data as input, we aim to learn a distribution involving all the classes input data can belong to, while keeping the data decentralized in each client storage. Our key contribution to this end is a new decentralized approach for learning GANs from non-iid data called Forgiver-First Update (F2U), which a) asks clients to train an individual discriminator with their own data and b) updates a generator to fool the most `forgiving' discriminators who deem generated samples as the most real. Our theoretical analysis proves that this updating strategy allows the decentralized GAN to achieve a generator's distribution with all the input classes as its global optimum based on f-divergence minimization. Moreover, we propose a relaxed version of F2U called Forgiver-First Aggregation (F2A) that performs well in practice, which adaptively aggregates the discriminators while emphasizing forgiving ones. Our empirical evaluations with image generation tasks demonstrated the effectiveness of our approach over state-of-the-art decentralized learning methods.