mlsanityData to 14 Oct 2026

A Large-Dimensional Analysis of Least-Squares Federated Learning via Generalized Total Variation Minimization

arXiv 2610.20075 ↗First posted 14 Oct 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning over decentralized networks inherently trades off the suppression of local finite-sample noise through collaboration against the bias that collaboration introduces between heterogeneous nodes. We study this tension by performing a large-dimensional analysis of Generalized Total Variation Minimization with least-squares models. Using tools from random matrix theory, we show that Lipschitz observations of the global GTVMin solution concentrate in large dimensions, universally for any underlying network structure. We provide exact analytic expressions for the asymptotically precise performance under both i.i.d. and non-i.i.d. Gaussian mixture distributions on a fixed network. For the latter, these expressions quantify how regularization balances local adaptation against collaboration. Our theoretical results are confirmed by experiments on synthetic and real-world data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

28 of 40 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2017Federated Multi-Task Learningcited by 672
  5. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  8. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  11. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  12. 2021Towards Personalized Federated Learningcited by 464
16 more
  1. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  2. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  3. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  4. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  5. 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 148
  6. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  7. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 76
  8. 2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 54
  9. 2015Network Lasso: Clustering and Optimization in Large Graphscited by 43
  10. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 36
  11. 2014On the Optimality of Averaging in Distributed Statistical Learningcited by 15
  12. 2019High Dimensional Classification via Regularized and Unregularized Empirical Risk Minimization: Precise Error and Optimal Losscited by 9
  13. 2021Federated Asymptotics: a model to compare federated learning algorithmscited by 8
  14. 2017A Large Dimensional Analysis of Least Squares Support Vector Machinescited by 7
  15. 2017A Fixed-Point of View on Gradient Methods for Big Datacited by 3
  16. 2025Federated Learning: From Theory to Practicecited by 3

Built on by

Not cited yet.