# FedBone: Towards Large-Scale Federated Multi-Task Learning

arXiv: https://arxiv.org/abs/2306.17465 · HTML: https://mlsanity.com/paper/2306.17465/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 71 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 30 of 42)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [Federated Learning with Buffered Asynchronous Aggregation](https://mlsanity.com/paper/2106.06639/) (2021)
- [FedKD: Communication Efficient Federated Learning via Knowledge Distillation](https://mlsanity.com/paper/2108.13323/) (2021)
- [FedVision: An Online Visual Object Detection Platform Powered by Federated Learning](https://mlsanity.com/paper/2001.06202/) (2020)
- [Overcoming Forgetting in Federated Learning on Non-IID Data](https://mlsanity.com/paper/1910.07796/) (2019)

## Built on by (most cited first, 14 of 14)

- [Ten Challenging Problems in Federated Foundation Models](https://mlsanity.com/paper/2502.12176/) (2025)
- [FedHCA2: Towards Hetero-Client Federated Multi-Task Learning](https://mlsanity.com/paper/2311.13250/) (2023)
- [FedPFT: Federated Proxy Fine-Tuning of Foundation Models](https://mlsanity.com/paper/2404.11536/) (2024)
- [Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2503.16585/) (2025)
- [Feasibility of Federated Learning from Client Databases with Different Brain Diseases and MRI Modalities](https://mlsanity.com/paper/2406.11636/) (2024)
- [Federated Communication-Efficient Multi-Objective Optimization](https://mlsanity.com/paper/2410.16398/) (2024)
- [Toward Enhancing Representation Learning in Federated Multi-Task Settings](https://mlsanity.com/paper/2602.01626/) (2026)
- [Fine-tuning Multimodal Transformers on Edge: A Parallel Split Learning Approach](https://mlsanity.com/paper/2502.06355/) (2025)
- [UniFed: A Universal Federation of a Mixture of Highly Heterogeneous Medical Image Classification Tasks](https://mlsanity.com/paper/2408.07075/) (2024)
- [ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems](https://mlsanity.com/paper/2501.10347/) (2025)
- [From Federated Learning to X-Learning: Breaking the Barriers of Decentrality Through Random Walks](https://mlsanity.com/paper/2509.03709/) (2025)
- [MuCALD-SplitFed: Causal-Latent Diffusion for Privacy-Preserving Multi-Task Split-Federated Medical Image Segmentation](https://mlsanity.com/paper/2605.04108/) (2026)
- [Hierarchical Multi-Task Federated Learning in VANETs](https://mlsanity.com/paper/2608.08111/) (2026)
- [Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication](https://mlsanity.com/paper/2608.15256/) (2026)

## Abstract (arXiv)

Heterogeneous federated multi-task learning (HFMTL) is a federated learning technique that combines heterogeneous tasks of different clients to achieve more accurate, comprehensive predictions. In real-world applications, visual and natural language tasks typically require large-scale models to extract high-level abstract features. However, large-scale models cannot be directly applied to existing federated multi-task learning methods. Existing HFML methods also disregard the impact of gradient conflicts on multi-task optimization during the federated aggregation process. In this work, we propose an innovative framework called FedBone, which enables the construction of large-scale models with better generalization from the perspective of server-client split learning and gradient projection. We split the entire model into two components: a large-scale general model (referred to as the general model) on the cloud server and multiple task-specific models (referred to as the client model) on edge clients, solving the problem of insufficient computing power on edge clients. The conflicting gradient projection technique is used to enhance the generalization of the large-scale general model between different tasks. The proposed framework is evaluated on two benchmark datasets and a real ophthalmic dataset. Comprehensive results demonstrate that FedBone efficiently adapts to heterogeneous local tasks of each client and outperforms existing federated learning algorithms in most dense prediction and classification tasks with off-the-shelf computational resources on the client side.