# Decentralized EM Algorithm for Gaussian Mixtures under Data Heterogeneity and Partial Labeling

arXiv: https://arxiv.org/abs/2411.05591 · HTML: https://mlsanity.com/paper/2411.05591/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 8 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Can AI help in screening Viral and COVID-19 pneumonia?](https://mlsanity.com/paper/2003.13145/) (2020)
- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization](https://mlsanity.com/paper/1404.6264/) (2014)
- [On the Convergence of Decentralized Gradient Descent](https://mlsanity.com/paper/1310.7063/) (2013)
- [Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification](https://mlsanity.com/paper/2012.03173/) (2020)
- [Decentralized Federated Learning: Balancing Communication and Computing Costs](https://mlsanity.com/paper/2107.12048/) (2021)
- [Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging](https://mlsanity.com/paper/2205.08576/) (2022)
- [Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence](https://mlsanity.com/paper/2111.09461/) (2021)
- [Network Gradient Descent Algorithm for Decentralized Federated Learning](https://mlsanity.com/paper/2205.08364/) (2022)

## Abstract (arXiv)

We systematically study several network-based Expectation-Maximization (EM) algorithms for the Gaussian mixture model within decentralized federated learning (DFL). Our theoretical investigation shows that directly extending the classic EM algorithm to DFL leads to a biased estimator when data are heterogeneously distributed across sites. To address this, we introduce a momentum network EM (MNEM) algorithm, which integrates information from both current and historical estimators from previous DFL iterations. We further develop a semi-supervised MNEM (semi-MNEM) algorithm, which utilizes information provided by partially labeled data. Rigorous theoretical analysis demonstrates that the MNEM estimator can achieve the same asymptotic efficiency as the whole-sample estimator under appropriate regularity conditions, even with heterogeneous data. Moreover, the semi-MNEM estimator significantly improves the convergence speed of the MNEM algorithm, even if different mixture components are poorly separated. Extensive simulations are conducted, and a widely used chest X-ray dataset is analyzed to demonstrate the finite-sample performance of the proposed methods.