mlsanityData to 7 Oct 2026

Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings

arXiv 2306.07959 ↗First posted 13 Jun 2023cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

In several practical applications of federated learning (FL), the clients are highly heterogeneous in terms of both their data and compute resources, and therefore enforcing the same model architecture for each client is very limiting. Moreover, the need for uncertainty quantification and data privacy constraints are often particularly amplified for clients that have limited local data. This paper presents a unified FL framework to simultaneously address all these constraints and concerns, based on training customized local Bayesian models that learn well even in the absence of large local datasets. A Bayesian framework provides a natural way of incorporating supervision in the form of prior distributions. We use priors in the functional (output) space of the networks to facilitate collaboration across heterogeneous clients. Moreover, formal differential privacy guarantees are provided for this framework. Experiments on standard FL datasets demonstrate that our approach outperforms strong baselines in both homogeneous and heterogeneous settings and under strict privacy constraints, while also providing characterizations of model uncertainties.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 65 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

45 of 65 references.

References · 45

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  3. 2018LEAF: A Benchmark for Federated Settingscited by 780
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2017Federated Multi-Task Learningcited by 669
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2017Renyi Differential Privacycited by 643
  8. 2020Federated Learning with Matched Averagingcited by 564
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  11. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
33 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  3. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  4. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  5. 2013The Composition Theorem for Differential Privacycited by 290
  6. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  7. 2019Model Fusion via Optimal Transportcited by 217
  8. 2021Personalized Federated Learning using Hypernetworkscited by 210
  9. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  10. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  11. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  12. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  13. 2019Functional Variational Bayesian Neural Networkscited by 127
  14. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  15. 2021Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Modelscited by 106
  16. 2016Stochastic Variational Deep Kernel Learningcited by 92
  17. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  18. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  19. 2021Personalized Federated Learning with Gaussian Processescited by 76
  20. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  21. 2019The Privacy Blanket of the Shuffle Modelcited by 73
  22. 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 72
  23. 2019Differentially Private Model Publishing for Deep Learningcited by 69
  24. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  25. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  26. 2019Federated Learning with Bayesian Differential Privacycited by 60
  27. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
  28. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  29. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  30. 2021GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learningcited by 26
  31. 2019Scalable and Differentially Private Distributed Aggregation in the Shuffled Modelcited by 20
  32. 2021WAFFLE: Weighted Averaging for Personalized Federated Learningcited by 9
  33. 2022How to Combine Variational Bayesian Networks in Federated Learningcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimationcited by 9
  2. 2024Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Reviewcited by 2