mlsanityData to 7 Oct 2026

Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomials

arXiv 2406.02416 ↗First posted 4 Jun 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

In practice, training using federated learning can be orders of magnitude slower than standard centralized training. This severely limits the amount of experimentation and tuning that can be done, making it challenging to obtain good performance on a given task. Server-side proxy data can be used to run training simulations, for instance for hyperparameter tuning. This can greatly speed up the training pipeline by reducing the number of tuning runs to be performed overall on the true clients. However, it is challenging to ensure that these simulations accurately reflect the dynamics of the real federated training. In particular, the proxy data used for simulations often comes as a single centralized dataset without a partition into distinct clients, and partitioning this data in a naive way can lead to simulations that poorly reflect real federated training. In this paper we address the challenge of how to partition centralized data in a way that reflects the statistical heterogeneity of the true federated clients. We propose a fully federated, theoretically justified, algorithm that efficiently learns the distribution of the true clients and observe improved server-side simulations when using the inferred distribution to create simulated clients from the centralized data.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

26 of 36 references.

References · 26

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2020The Future of Digital Health with Federated Learningcited by 625
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  10. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  11. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
14 more
  1. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  3. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  4. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  5. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  6. 2019Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Recordscited by 70
  7. 2021SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Trainingcited by 45
  8. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  9. 2020Improving on-device speaker verification using federated learning with privacycited by 29
  10. 2022FLAIR: Federated Learning Annotated Image Repositorycited by 22
  11. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  12. 2024pfl-research: simulation framework for accelerating research in Private Federated Learningcited by 8
  13. 2022Personalized Federated Learning with Server-Side Informationcited by 6
  14. 2022Federated Learning with Server Learning: Enhancing Performance for Non-IID Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Trainingcited by 0
  2. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0
  3. 2026FedLBW: A Loss-Based Weighting Strategy for Federated Learning on Non-IID Data in Wireless Networkscited by 0