mlsanityData to 7 Oct 2026

A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularization

arXiv 2102.07148 ↗First posted 14 Feb 2021cs.LG cs.DCcited by 27

Not on the map. Why?

Abstract (arXiv)

Non-Independent and Identically Distributed (non- IID) data distribution among clients is considered as the key factor that degrades the performance of federated learning (FL). Several approaches to handle non-IID data such as personalized FL and federated multi-task learning (FMTL) are of great interest to research communities. In this work, first, we formulate the FMTL problem using Laplacian regularization to explicitly leverage the relationships among the models of clients for multi-task learning. Then, we introduce a new view of the FMTL problem, which in the first time shows that the formulated FMTL problem can be used for conventional FL and personalized FL. We also propose two algorithms FedU and dFedU to solve the formulated FMTL problem in communication-centralized and decentralized schemes, respectively. Theoretically, we prove that the convergence rates of both algorithms achieve linear speedup for strongly convex and sublinear speedup of order 1/2 for nonconvex objectives. Experimentally, we show that our algorithms outperform the algorithm FedAvg, FedProx, SCAFFOLD, and AFL in FL settings, MOCHA in FMTL settings, as well as pFedMe and Per-FedAvg in personalized FL settings.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 41 of its 71 references that are in the map. First posted on arXiv 14 Feb 2021.

Builds on

41 of 71 references.

References · 41

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2017Federated Multi-Task Learningcited by 669
29 more
  1. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  2. 2020The Future of Digital Health with Federated Learningcited by 625
  3. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  4. 2019Federated Learning with Personalization Layerscited by 533
  5. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  7. 2019Agnostic Federated Learningcited by 451
  8. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  9. 2020Adaptive Personalized Federated Learningcited by 353
  10. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  11. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  12. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  13. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  14. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  16. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  17. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  18. 2019Unified Optimal Analysis of the (Stochastic) Gradient Methodcited by 81
  19. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  20. 2018A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updatescited by 47
  21. 2015Network Lasso: Clustering and Optimization in Large Graphscited by 42
  22. 2018Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimizationcited by 34
  23. 2020Local Graph Clustering with Network Lassocited by 16
  24. 2018Learning over Multitask Graphs -- Part I: Stability Analysiscited by 12
  25. 2019Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noisecited by 9
  26. 2019Localized Linear Regression in Networked Datacited by 8
  27. 2019A Distributed Method for Fitting Laplacian Regularized Stratified Modelscited by 7
  28. 2018Learning over Multitask Graphs -- Part II: Performance Analysiscited by 5
  29. 2020Eigen-Stratified Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 4; 2024: 8; 2025: 6; 2026: 3.

27 papers.

Sort:

  1. 2022Federated Graph Neural Networks: Overview, Techniques and Challengescited by 52
  2. 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
  3. 2022IOP-FL: Inside-Outside Personalization for Federated Medical Image Segmentationcited by 21
  4. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  5. 2023FedICT: Federated Multi-task Distillation for Multi-access Edge Computingcited by 17
  6. 2022Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imagingcited by 14
  7. 2022Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applicationscited by 14
  8. 2025Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approachcited by 10
  9. 2025A Comprehensive Data-centric Overview of Federated Graph Learningcited by 7
  10. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  11. 2024Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradientscited by 3
  12. 2024MIRA: A Method of Federated MultI-Task Learning for LaRge LAnguage Modelscited by 3
15 more
  1. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 2
  2. 2022Graph Federated Learning with Hidden Representation Sharingcited by 1
  3. 2023Joint Graph Learning and Model Fitting in Laplacian Regularized Stratified Modelscited by 1
  4. 2024Federated Learning in Healthcare: Model Misconducts, Security, Challenges, Applications, and Future Research Directions -- A Systematic Reviewcited by 1
  5. 2025FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworkscited by 1
  6. 2023Global Layers: Non-IID Tabular Federated Learningcited by 0
  7. 2024HF-Fed: Hierarchical based customized Federated Learning Framework for X-Ray Imagingcited by 0
  8. 2024Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Informationcited by 0
  9. 2024A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Datacited by 0
  10. 2025FedVSR: Towards Model-Agnostic Federated Learning in Video Super-Resolutioncited by 0
  11. 2025Personalized Federated Learning under Model Dissimilarity Constraintscited by 0
  12. 2025Learning to Collaborate Over Graphs: A Selective Federated Multi-Task Learning Approachcited by 0
  13. 2026Sheaf-Based Federated Representation Learningcited by 0
  14. 2026Transfer Learning in Nonparametric Regression with Deep ReLU Networkscited by 0
  15. 2026A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimizationcited by 0