mlsanityData to 7 Oct 2026

Multi-Model Federated Learning with Provable Guarantees

arXiv 2207.04330 ↗First posted 9 Jul 2022cs.LG cs.DC math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a variant of distributed learning where edge devices collaborate to learn a model without sharing their data with the central server or each other. We refer to the process of training multiple independent models simultaneously in a federated setting using a common pool of clients as multi-model FL. In this work, we propose two variants of the popular FedAvg algorithm for multi-model FL, with provable convergence guarantees. We further show that for the same amount of computation, multi-model FL can have better performance than training each model separately. We supplement our theoretical results with experiments in strongly convex, convex, and non-convex settings.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 9 Jul 2022.

Builds on

24 of 29 references.

References · 24

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  8. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  9. 2019Fair Resource Allocation in Federated Learningcited by 389
  10. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  11. 2020Adaptive Personalized Federated Learningcited by 353
  12. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
12 more
  1. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  2. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  3. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  4. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
  5. 2018Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDcited by 76
  6. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  7. 2015Convergence Rate of Incremental Gradient and Newton Methodscited by 26
  8. 2020FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cyclingcited by 26
  9. 2020Federated Residual Learningcited by 23
  10. 2019Communication-Efficient Local Decentralized SGD Methodscited by 21
  11. 2020Multi-Level Local SGD for Heterogeneous Hierarchical Networkscited by 9
  12. 2022Multi-Model Federated Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 4.

5 papers.

  1. 2024COALA: A Practical and Vision-Centric Federated Learning Platformcited by 7
  2. 2024Fair Concurrent Training of Multiple Models in Federated Learningcited by 3
  3. 2024FedAST: Federated Asynchronous Simultaneous Trainingcited by 3
  4. 2022Sequential Gradient Coding For Straggler Mitigationcited by 1
  5. 2024CELLM: An Efficient Communication in Large Language Models Training for Federated Learningcited by 0