Task Diversity in Bayesian Federated Learning: Simultaneous Processing of Classification and Regression
Not on the map. Why?
Abstract (arXiv)
This work addresses a key limitation in current federated learning approaches, which predominantly focus on homogeneous tasks, neglecting the task diversity on local devices. We propose a principled integration of multi-task learning using multi-output Gaussian processes (MOGP) at the local level and federated learning at the global level. MOGP handles correlated classification and regression tasks, offering a Bayesian non-parametric approach that naturally quantifies uncertainty. The central server aggregates the posteriors from local devices, updating a global MOGP prior redistributed for training local models until convergence. Challenges in performing posterior inference on local devices are addressed through the P'olya-Gamma augmentation technique and mean-field variational inference, enhancing computational efficiency and convergence rate. Experimental results on both synthetic and real data demonstrate superior predictive performance, OOD detection, uncertainty calibration and convergence rate, highlighting the method's potential in diverse applications. Our code is publicly available at https://github.com/JunliangLv/taskdiversityBFL.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 71 references that are in the map. First posted on arXiv 14 Dec 2024.
Builds on
41 of 71 references.
References · 41
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2016Variational Inference: A Review for Statisticianscited by 1,472
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
- 2020Designing Network Design Spacescited by 960
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
- 2017Federated Multi-Task Learningcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2018Local SGD Converges Fast and Communicates Littlecited by 605
29 more
- 2019Federated Learning with Personalization Layerscited by 541
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
- 2018End-to-End Multi-Task Learning with Attentioncited by 534
- 2015Deep Kernel Learningcited by 445
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
- 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
- 2014Scalable Variational Gaussian Process Classificationcited by 228
- 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 210
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
- 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
- 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
- 2019On the Convergence of Local Descent Methods in Federated Learningcited by 153
- 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
- 2021Personalized Federated Learning with Gaussian Processescited by 76
- 2019Variational Federated Multi-Task Learningcited by 69
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2019Federated Learning with Bayesian Differential Privacycited by 60
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
- 2020Bayesian Few-Shot Classification with One-vs-Each P'olya-Gamma Augmented Gaussian Processescited by 32
- 2018Heterogeneous Multi-output Gaussian Process Predictioncited by 26
- 2018Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentationcited by 17
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2019Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentationcited by 16
- 2020Multi-output Gaussian Process Modulated Poisson Processes for Event Predictioncited by 5
- 2023Heterogeneous Multi-Task Gaussian Cox Processescited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.