mlsanityData to 14 Oct 2026

Global Convergence of Federated Learning for Mixed Regression

arXiv 2206.07279 ↗First posted 15 Jun 2022cs.LG cs.DCcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper studies the problem of model training under Federated Learning when clients exhibit cluster structure. We contextualize this problem in mixed regression, where each client has limited local data generated from one of k unknown regression models. We design an algorithm that achieves global convergence from any initialization, and works even when local data volume is highly unbalanced -- there could exist clients that contain O(1) data points only. Our algorithm first runs moment descent on a few anchor clients (each with tildeOmega(k) data points) to obtain coarse model estimates. Then each client alternately estimates its cluster labels and refines the model estimates based on FedAvg or FedProx. A key innovation in our analysis is a uniform estimate on the clustering errors, which we prove by bounding the VC dimension of general polynomial concept classes based on the theory of algebraic geometry.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 22 of its 58 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

22 of 58 references.

References · 22

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2017Federated Multi-Task Learningcited by 672
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  11. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  12. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
10 more
  1. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  2. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  3. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  4. 2020Provable Meta-Learning of Linear Representationscited by 161
  5. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 149
  6. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
  7. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 90
  8. 2020Meta-learning for mixed linear regressioncited by 44
  9. 2018Learning Mixtures of Linear Regressions with Nearly Optimal Complexitycited by 37
  10. 2021A Non-parametric View of FedAvg and FedProx: Beyond Stationary Pointscited by 13

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 2.

6 papers.

  1. 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 27
  2. 2023Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithmscited by 7
  3. 2024Learning with Shared Representations: Statistical Rates and Efficient Algorithmscited by 1
  4. 2025Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimizationcited by 1
  5. 2024Fast convergence of a Federated Expectation-Maximization Algorithmcited by 0
  6. 2025Federated Independent Component Analysis via Spectral Alignment and Robust Aggregationcited by 0