mlsanityData to 14 Oct 2026

FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification

arXiv 2404.15657 ↗First posted 24 Apr 2024cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

While deep neural networks (DNNs) based personalized federated learning (PFL) is demanding for addressing data heterogeneity and shows promising performance, existing methods for federated learning (FL) suffer from efficient systematic uncertainty quantification. The Bayesian DNNs-based PFL is usually questioned of either over-simplified model structures or high computational and memory costs. In this paper, we introduce FedSI, a novel Bayesian DNNs-based subnetwork inference PFL framework. FedSI is simple and scalable by leveraging Bayesian methods to incorporate systematic uncertainties effectively. It implements a client-specific subnetwork inference mechanism, selects network parameters with large variance to be inferred through posterior distributions, and fixes the rest as deterministic ones. FedSI achieves fast and scalable inference while preserving the systematic uncertainties to the fullest extent. Extensive experiments on three different benchmark datasets demonstrate that FedSI outperforms existing Bayesian and non-Bayesian FL baselines in heterogeneous FL scenarios.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 55 of its 95 references that are in the map. First posted on arXiv 24 Apr 2024.

Builds on

55 of 95 references.

References · 55

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2019Advances and Open Problems in Federated Learningcited by 3,046
  7. 2016Concrete Problems in AI Safetycited by 1,642
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2019Federated Learning with Personalization Layerscited by 541
38 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  2. 2021Towards Personalized Federated Learningcited by 464
  3. 2014New insights and perspectives on the natural gradient methodcited by 443
  4. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  5. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  6. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  7. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  8. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  9. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  10. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  11. 2022Federated Learning with Partial Model Personalizationcited by 114
  12. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  13. 2019Subspace Inference for Bayesian Deep Learningcited by 95
  14. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 87
  15. 2021GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophilycited by 84
  16. 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 80
  17. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  18. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  19. 2021Personalized Federated Learning with Gaussian Processescited by 76
  20. 2020Faster Wasserstein Distance Estimation with the Sinkhorn Divergencecited by 73
  21. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  22. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  23. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  24. 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
  25. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  26. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  27. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  28. 2021A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularizationcited by 27
  29. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  30. 2023Bayesian Federated Learning: A Surveycited by 22
  31. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  32. 2023FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clusteringcited by 15
  33. 2022Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imagingcited by 14
  34. 2021Function-space Inference with Sparse Implicit Processescited by 13
  35. 2021Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspectivecited by 9
  36. 2022Personalized Federated Learning via Heterogeneous Modular Networkscited by 8
  37. 2023Free-Form Variational Inference for Gaussian Process State-Space Modelscited by 7
  38. 2019Rectangular Bounding Processcited by 6

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Federated Neural Nonparametric Point Processescited by 3
  2. 2025UFGraphFR: Graph Federation Recommendation System based on User Text description featurescited by 1
  3. 2026Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrievalcited by 1
  4. 2026FedUAF: Uncertainty-Aware Fusion with Reliability-Guided Aggregation for Multimodal Federated Sentiment Analysiscited by 0