# Federated Variational Inference for Bayesian Mixture Models

arXiv: https://arxiv.org/abs/2502.12684 · HTML: https://mlsanity.com/paper/2502.12684/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 74 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Heterogeneity for the Win: One-Shot Federated Clustering](https://mlsanity.com/paper/2103.00697/) (2021)
- [Personalized Federated Learning with Gaussian Processes](https://mlsanity.com/paper/2106.15482/) (2021)
- [Personalized Federated Learning under Mixture of Distributions](https://mlsanity.com/paper/2305.01068/) (2023)
- [Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)](https://mlsanity.com/paper/2201.07316/) (2022)
- [Bayesian Federated Learning: A Survey](https://mlsanity.com/paper/2304.13267/) (2023)
- [Variational Tempering](https://mlsanity.com/paper/1411.1810/) (2014)
- [Federated Learning with Uncertainty via Distilled Predictive Distributions](https://mlsanity.com/paper/2206.07562/) (2022)
- [Model-based clustering based on sparse finite Gaussian mixtures](https://mlsanity.com/paper/1606.06828/) (2016)
- [Federated K-means Clustering](https://mlsanity.com/paper/2310.01195/) (2023)
- [Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms](https://mlsanity.com/paper/2310.15330/) (2023)
- [Overfitting Bayesian Mixture Models with an Unknown Number of Components](https://mlsanity.com/paper/1502.05427/) (2015)
- [Variational consensus Monte Carlo](https://mlsanity.com/paper/1506.03074/) (2015)
- [Streaming, Distributed Variational Inference for Bayesian Nonparametrics](https://mlsanity.com/paper/1510.09161/) (2015)
- [CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data](https://mlsanity.com/paper/1512.01272/) (2015)
- [Scalable Bayesian Nonparametric Clustering and Classification](https://mlsanity.com/paper/1806.02670/) (2018)
- [Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering](https://mlsanity.com/paper/2303.04345/) (2023)
- [Big Learning with Bayesian Methods](https://mlsanity.com/paper/1411.6370/) (2014)
- [Identifying cancer subtypes in glioblastoma by combining genomic, transcriptomic and epigenomic data](https://mlsanity.com/paper/1304.3577/) (2013)
- [Embarrassingly Parallel Variational Inference in Nonconjugate Models](https://mlsanity.com/paper/1510.04163/) (2015)
- [Consensus Monte Carlo for Random Subsets using Shared Anchors](https://mlsanity.com/paper/1906.12309/) (2019)
- [Distributed Bayesian clustering using finite mixture of mixtures](https://mlsanity.com/paper/2003.13936/) (2020)
- [VICatMix: variational Bayesian clustering and variable selection for discrete biomedical data](https://mlsanity.com/paper/2406.16227/) (2024)

## Built on by (most cited first, 1 of 1)

- [Variational Consensus Monte Carlo for Bayesian Mixture](https://mlsanity.com/paper/2606.19643/) (2026)

## Abstract (arXiv)

We present a federated learning approach for Bayesian model-based clustering of large-scale binary and categorical datasets. We introduce a principled 'divide and conquer' inference procedure using variational inference with local merge and delete moves within batches of the data in parallel, followed by 'global' merge moves across batches to find global clustering structures. We show that these merge moves require only summaries of the data in each batch, enabling federated learning across local nodes without requiring the full dataset to be shared. Empirical results on simulated and benchmark datasets demonstrate that our method performs well in comparison to existing clustering algorithms. We validate the practical utility of the method by applying it to large scale electronic health record (EHR) data.