mlsanityData to 14 Oct 2026

Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning

arXiv 2509.10132 ↗First posted 12 Sep 2025cs.LG cs.IT cs.NIcited by 0

Not on the map. Why?

Abstract (arXiv)

Bayesian Federated Learning (BFL) combines uncertainty modeling with decentralized training, enabling the development of personalized and reliable models under data heterogeneity and privacy constraints. Existing approaches typically rely on Markov Chain Monte Carlo (MCMC) sampling or variational inference, often incorporating personalization mechanisms to better adapt to local data distributions. In this work, we propose an information-geometric projection framework for personalization in parametric BFL. By projecting the global model onto a neighborhood of the user's local model, our method enables a tunable trade-off between global generalization and local specialization. Under mild assumptions, we show that this projection step is equivalent to computing a barycenter on the statistical manifold, allowing us to derive closed-form solutions and achieve cost-free personalization. We apply the proposed approach to a variational learning setup using the Improved Variational Online Newton (IVON) optimizer and extend its application to general aggregation schemes in BFL. Empirical evaluations under heterogeneous data distributions confirm that our method effectively balances global and local performance with minimal computational overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 61 of its 77 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

61 of 77 references.

References · 61

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  6. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2018LEAF: A Benchmark for Federated Settingscited by 792
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2019Federated Learning with Personalization Layerscited by 541
38 more
  1. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  2. 2021Towards Personalized Federated Learningcited by 464
  3. 2015Optimal Transport for Domain Adaptationcited by 463
  4. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  5. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  6. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  7. 2023Continual Pre-training of Language Modelscited by 198
  8. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  9. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  10. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  11. 2020Fair Regression with Wasserstein Barycenterscited by 100
  12. 2021The Bayesian Learning Rulecited by 99
  13. 2024Model Stock: All we need is just a few fine-tuned modelscited by 92
  14. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  15. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  16. 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
  17. 2019Variational Federated Multi-Task Learningcited by 69
  18. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  19. 2024Variational Learning is Effective for Large Deep Networkscited by 65
  20. 2021Federated Learning with Fair Averagingcited by 61
  21. 2021Towards Fairness-Aware Federated Learningcited by 58
  22. 2020Federated Learning Meets Multi-objective Optimizationcited by 53
  23. 2020Federated learning with class imbalance reductioncited by 52
  24. 2018How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairnesscited by 48
  25. 2014Multi-marginal optimal transport: theory and applicationscited by 41
  26. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  27. 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 35
  28. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  29. 2022Fair learning with Wasserstein barycenters for non-decomposable performance measurescited by 24
  30. 2022Fusion of Probability Density Functionscited by 22
  31. 2023Bayesian Federated Learning: A Surveycited by 22
  32. 2022Proportional Fairness in Federated Learningcited by 21
  33. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  34. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  35. 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
  36. 2021Entropic Gromov-Wasserstein between Gaussian Distributionscited by 12
  37. 2022An Optimal Transport Approach to Personalized Federated Learningcited by 12
  38. 2022Federated Learning with Uncertainty via Distilled Predictive Distributionscited by 12

50 most cited of 61 All 61 (JSON)

Built on by

Not cited yet.