mlsanityData to 7 Oct 2026

Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Study

arXiv 2402.12876 ↗First posted 20 Feb 2024cs.LG cs.CR cs.DCcited by 2

Not on the map. Why?

Abstract (arXiv)

The innovative Federated Multi-Task Learning (FMTL) approach consolidates the benefits of Federated Learning (FL) and Multi-Task Learning (MTL), enabling collaborative model training on multi-task learning datasets. However, a comprehensive evaluation method, integrating the unique features of both FL and MTL, is currently absent in the field. This paper fills this void by introducing a novel framework, FMTL-Bench, for systematic evaluation of the FMTL paradigm. This benchmark covers various aspects at the data, model, and optimization algorithm levels, and comprises seven sets of comparative experiments, encapsulating a wide array of non-independent and identically distributed (Non-IID) data partitioning scenarios. We propose a systematic process for comparing baselines of diverse indicators and conduct a case study on communication expenditure, time, and energy consumption. Through our exhaustive experiments, we aim to provide valuable insights into the strengths and limitations of existing baseline methods, contributing to the ongoing discourse on optimal FMTL application in practical scenarios. The source code can be found on https://github.com/youngfish42/FMTL-Benchmark .

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 66 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

34 of 66 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2021Vision Transformers for Dense Predictioncited by 1,789
  10. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  11. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  12. 2017Federated Multi-Task Learningcited by 669
22 more
  1. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  2. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  3. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  4. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  5. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  6. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  7. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  8. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  9. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  10. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  11. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  12. 2021FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendationcited by 97
  13. 2022InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understandingcited by 89
  14. 2021MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluationcited by 88
  15. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  16. 2020Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computingcited by 52
  17. 2020Can Federated Learning Save The Planet?cited by 52
  18. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  19. 2020The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systemscited by 31
  20. 2021Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligencecited by 18
  21. 2023MAS: Towards Resource-Efficient Federated Multiple-Task Learningcited by 9
  22. 2023Handling Data Heterogeneity via Architectural Design for Federated Visual Recognitioncited by 8

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generationcited by 7
  2. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4