mlsanityData to 14 Oct 2026

CoBo: Collaborative Learning via Bilevel Optimization

arXiv 2409.05539 ↗First posted 9 Sep 2024cs.LG cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

Collaborative learning is an important tool to train multiple clients more effectively by enabling communication among clients. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model client-selection and model-training as two interconnected optimization problems, proposing a novel bilevel optimization problem for collaborative learning. We introduce CoBo, a scalable and elastic, SGD-type alternating optimization algorithm that efficiently addresses these problem with theoretical convergence guarantees. Empirically, CoBo achieves superior performance, surpassing popular personalization algorithms by 9.3% in accuracy on a task with high heterogeneity, involving datasets distributed among 80 clients.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

28 of 42 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2017Federated Multi-Task Learningcited by 672
  7. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  11. 2021Towards Personalized Federated Learningcited by 464
  12. 2019Agnostic Federated Learningcited by 456
16 more
  1. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  2. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  3. 2019Lower Bounds for Non-Convex Stochastic Optimizationcited by 307
  4. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  5. 2020Survey of Personalization Techniques for Federated Learningcited by 145
  6. 2020Reinforcement Learning via Fenchel-Rockafellar Dualitycited by 100
  7. 2020Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learningcited by 47
  8. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 36
  9. 2021BiAdam: Fast Adaptive Bilevel Optimization Methodscited by 33
  10. 2022Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Datacited by 32
  11. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  12. 2023Provably Personalized and Robust Federated Learningcited by 11
  13. 2024First-Order Methods for Linearly Constrained Bilevel Optimizationcited by 10
  14. 2024Personalized Collaborative Fine-Tuning for On-Device Large Language Modelscited by 9
  15. 2024Unravelling in Collaborative Learningcited by 6
  16. 2021Federated Expectation Maximization with heterogeneity mitigation and variance reductioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Adaptive collaboration for online personalized distributed learning with heterogeneous clientscited by 2
  2. 2026DUET: Decentralized Bilevel Optimization without Lower-Level Strong Convexitycited by 1
  3. 2025Personalized Federated Learning under Model Dissimilarity Constraintscited by 0
  4. 2025Enhancing Privacy in Decentralized Min-Max Optimization: A Differentially Private Approachcited by 0
  5. 2026Adaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddingscited by 0