# FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference

arXiv: https://arxiv.org/abs/2605.08760 · HTML: https://mlsanity.com/paper/2605.08760/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 19 of 19)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [FedGAN: Federated Generative Adversarial Networks for Distributed Data](https://mlsanity.com/paper/2006.07228/) (2020)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learning](https://mlsanity.com/paper/2111.08211/) (2021)
- [Personalized Federated Learning under Mixture of Distributions](https://mlsanity.com/paper/2305.01068/) (2023)
- [An Expectation-Maximization Perspective on Federated Learning](https://mlsanity.com/paper/2111.10192/) (2021)
- [FedLP: Layer-wise Pruning Mechanism for Communication-Computation Efficient Federated Learning](https://mlsanity.com/paper/2303.06360/) (2023)
- [How global observation works in Federated Learning: Integrating vertical training into Horizontal Federated Learning](https://mlsanity.com/paper/2112.01039/) (2021)
- [Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering](https://mlsanity.com/paper/2303.04345/) (2023)
- [ISFL: Federated Learning for Non-i.i.d. Data with Local Importance Sampling](https://mlsanity.com/paper/2210.02119/) (2022)
- [How to train your VAE](https://mlsanity.com/paper/2309.13160/) (2023)
- [A Bayesian Framework for Clustered Federated Learning](https://mlsanity.com/paper/2410.15473/) (2024)
- [FedNC: A Secure and Efficient Federated Learning Method with Network Coding](https://mlsanity.com/paper/2305.03292/) (2023)

## Abstract (arXiv)

Federated Learning (FL) facilitates collaborative model training across decentralized clients while preserving data privacy by avoiding raw data exchange. Despite its potential, FL performance is often compromised by data heterogeneity across clients. To address this, Clustered Federated Learning (CFL) groups clients with similar data distributions to improve model performance, but constrained by intra-cluster heterogeneity. Conversely, Personalized Federated Learning (PFL) tailors models to individual clients, but usually neglects the underlying structural similarities among clients. In this work, we investigate a probabilistic mixture (PM) scenario, where each client's local data distribution is modeled as a convex combination of several shared inherent distributions. To effectively model this structure, we propose FedGMI, a framework that utilizes Variational Autoencoders (VAEs) as generative density estimators to represent these inherent distributions and infer the mixture components of clients' local data distributions. This approach enables structured personalization without sacrificing the benefits of collaborative learning. Extensive experiments demonstrate that FedGMI effectively characterizes and discriminate the inherent distributions, as well as accurately estimates mixture proportions. Furthermore, FedGMI maintains robust performance even under communication cost constraints.