mlsanityData to 7 Oct 2026

MAS: Towards Resource-Efficient Federated Multiple-Task Learning

arXiv 2307.11285 ↗First posted 21 Jul 2023cs.LG cs.CV cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is an emerging distributed machine learning method that empowers in-situ model training on decentralized edge devices. However, multiple simultaneous FL tasks could overload resource-constrained devices. In this work, we propose the first FL system to effectively coordinate and train multiple simultaneous FL tasks. We first formalize the problem of training simultaneous FL tasks. Then, we present our new approach, MAS (Merge and Split), to optimize the performance of training multiple simultaneous FL tasks. MAS starts by merging FL tasks into an all-in-one FL task with a multi-task architecture. After training for a few rounds, MAS splits the all-in-one FL task into two or more FL tasks by using the affinities among tasks measured during the all-in-one training. It then continues training each split of FL tasks based on model parameters from the all-in-one training. Extensive experiments demonstrate that MAS outperforms other methods while reducing training time by 2x and reducing energy consumption by 40%. We hope this work will inspire the community to further study and optimize training simultaneous FL tasks.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 47 of its 68 references that are in the map. First posted on arXiv 21 Jul 2023.

Builds on

47 of 68 references.

References · 47

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2017A Survey on Multi-Task Learningcited by 838
  10. 2019Towards Federated Learning at Scale: System Designcited by 826
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  12. 2018How To Backdoor Federated Learningcited by 745
35 more
  1. 2017Federated Multi-Task Learningcited by 669
  2. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  3. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  4. 2020Federated Learning with Matched Averagingcited by 564
  5. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  6. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  7. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  8. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  9. 2020Adaptive Personalized Federated Learningcited by 353
  10. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  11. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  12. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  13. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  14. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  15. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  16. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  17. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 159
  18. 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
  19. 2020Learning to Branch for Multi-Task Learningcited by 114
  20. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  21. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  22. 2020TiFL: A Tier-based Federated Learning Systemcited by 95
  23. 2016Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classificationcited by 93
  24. 2019Branched Multi-Task Networks: Deciding What Layers To Sharecited by 76
  25. 2022Divergence-aware Federated Self-Supervised Learningcited by 71
  26. 2018Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotationscited by 51
  27. 2020Performance Optimization for Federated Person Re-identification via Benchmark Analysiscited by 50
  28. 2018A Modulation Module for Multi-task Learning with Applications in Image Retrievalcited by 49
  29. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 36
  30. 2016Integrated perception with recurrent multi-task neural networkscited by 24
  31. 2021EasyFL: A Low-code Federated Learning Platform For Dummiescited by 20
  32. 2018GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learningcited by 17
  33. 2022FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregationcited by 16
  34. 2023FedWon: Triumphing Multi-domain Federated Learning Without Normalizationcited by 11
  35. 2022Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysiscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 3; 2026: 1.

9 papers.

  1. 2025FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulationcited by 18
  2. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  3. 2024FedMef: Towards Memory-efficient Federated Dynamic Pruningcited by 14
  4. 2024COALA: A Practical and Vision-Centric Federated Learning Platformcited by 7
  5. 2025Many-Task Federated Fine-Tuning via Unified Task Vectorscited by 3
  6. 2025Redefining non-IID Data in Federated Learning for Computer Vision Tasks: Migrating from Labels to Embeddings for Task-Specific Data Distributionscited by 3
  7. 2024Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Studycited by 2
  8. 2024UniFed: A Universal Federation of a Mixture of Highly Heterogeneous Medical Image Classification Taskscited by 0
  9. 2026DART-FL: Burst-Aware Multitask Federated Learning under Dynamic Inference Demand at the Edgecited by 0