# Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomials

arXiv: https://arxiv.org/abs/2406.02416 · HTML: https://mlsanity.com/paper/2406.02416/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Federated Learning for Emoji Prediction in a Mobile Keyboard](https://mlsanity.com/paper/1906.04329/) (2019)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records](https://mlsanity.com/paper/1903.09296/) (2019)
- [SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training](https://mlsanity.com/paper/2106.01432/) (2021)
- [Improving on-device speaker verification using federated learning with privacy](https://mlsanity.com/paper/2008.02651/) (2020)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)
- [FLAIR: Federated Learning Annotated Image Repository](https://mlsanity.com/paper/2207.08869/) (2022)
- [PeFLL: Personalized Federated Learning by Learning to Learn](https://mlsanity.com/paper/2306.05515/) (2023)
- [pfl-research: simulation framework for accelerating research in Private Federated Learning](https://mlsanity.com/paper/2404.06430/) (2024)
- [Personalized Federated Learning with Server-Side Information](https://mlsanity.com/paper/2205.11044/) (2022)
- [Federated Learning with Server Learning: Enhancing Performance for Non-IID Data](https://mlsanity.com/paper/2210.02614/) (2022)

## Built on by (most cited first, 3 of 3)

- [FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training](https://mlsanity.com/paper/2605.02337/) (2026)
- [From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning](https://mlsanity.com/paper/2606.10595/) (2026)
- [FedLBW: A Loss-Based Weighting Strategy for Federated Learning on Non-IID Data in Wireless Networks](https://mlsanity.com/paper/2608.07007/) (2026)

## Abstract (arXiv)

In practice, training using federated learning can be orders of magnitude slower than standard centralized training. This severely limits the amount of experimentation and tuning that can be done, making it challenging to obtain good performance on a given task. Server-side proxy data can be used to run training simulations, for instance for hyperparameter tuning. This can greatly speed up the training pipeline by reducing the number of tuning runs to be performed overall on the true clients. However, it is challenging to ensure that these simulations accurately reflect the dynamics of the real federated training. In particular, the proxy data used for simulations often comes as a single centralized dataset without a partition into distinct clients, and partitioning this data in a naive way can lead to simulations that poorly reflect real federated training. In this paper we address the challenge of how to partition centralized data in a way that reflects the statistical heterogeneity of the true federated clients. We propose a fully federated, theoretically justified, algorithm that efficiently learns the distribution of the true clients and observe improved server-side simulations when using the inferred distribution to create simulated clients from the centralized data.