A One-shot Framework for Distributed Clustered Learning in Heterogeneous Environments
Not on the map. Why?
Abstract (arXiv)
The paper proposes a family of communication efficient methods for distributed learning in heterogeneous environments in which users obtain data from one of K different distributions. In the proposed setup, the grouping of users (based on the data distributions they sample), as well as the underlying statistical properties of the distributions, are apriori unknown. A family of One-shot Distributed Clustered Learning methods (ODCL-C) is proposed, parametrized by the set of admissible clustering algorithms C, with the objective of learning the true model at each user. The admissible clustering methods include K-means (KM) and convex clustering (CC), giving rise to various one-shot methods within the proposed family, such as ODCL-KM and ODCL-CC. The proposed one-shot approach, based on local computations at the users and a clustering based aggregation step at the server is shown to provide strong learning guarantees. In particular, for strongly convex problems it is shown that, as long as the number of data points per user is above a threshold, the proposed approach achieves order-optimal mean-squared error (MSE) rates in terms of the sample size. An explicit characterization of the threshold is provided in terms of problem parameters. The trade-offs with respect to selecting various clustering methods (ODCL-CC, ODCL-KM) are discussed and significant improvements over state-of-the-art are demonstrated. Numerical experiments illustrate the findings and corroborate the performance of the proposed methods.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 69 references that are in the map. First posted on arXiv 22 Sep 2022.
Builds on
24 of 69 references.
References · 24
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
- 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
- 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
- 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
- 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020Salvaging Federated Learning by Local Adaptationcited by 142
- 2019One-Shot Federated Learningcited by 124
12 more
- 2020Distilled One-Shot Federated Learningcited by 108
- 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
- 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
- 2013Splitting Methods for Convex Clusteringcited by 28
- 2018Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithmcited by 26
- 2019Simple and optimal high-probability bounds for strongly-convex stochastic gradient descentcited by 25
- 2021The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methodscited by 24
- 2019One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Themcited by 23
- 2017Empirical Risk Minimization for Stochastic Convex Optimization: O(1/n)- and O(1/n2)-type of Risk Boundscited by 15
- 2022Personalized Federated Learning via Convex Clusteringcited by 10
- 2022Gradient Based Clusteringcited by 5
- 2022FedBC: Calibrating Global and Local Models via Federated Learning Beyond Consensuscited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.
3 papers.