From Local SGD to Local Fixed-Point Methods for Federated Learning
Not on the map. Why?
Abstract (arXiv)
Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed-point algorithms. In this work we consider the generic problem of finding a fixed point of an average of operators, or an approximation thereof, in a distributed setting. Our work is motivated by the needs of federated learning. In this context, each local operator models the computations done locally on a mobile device. We investigate two strategies to achieve such a consensus: one based on a fixed number of local steps, and the other based on randomized computations. In both cases, the goal is to limit communication of the locally-computed variables, which is often the bottleneck in distributed frameworks. We perform convergence analysis of both methods and conduct a number of experiments highlighting the benefits of our approach.
Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Apr 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 17; 2022: 16; 2023: 22; 2024: 7; 2025: 7; 2026: 5.
74 papers.
Sort:
- 2021Federated Learning Based on Dynamic Regularizationcited by 555
- 2021A Field Guide to Federated Optimizationcited by 312
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
- 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
- 2021On Large-Cohort Training for Federated Learningcited by 73
- 2021Provably Secure Federated Learning against Malicious Clientscited by 60
- 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
- 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 53
- 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
- 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
- 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
38 more
- 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
- 2021Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivitycited by 39
- 2022On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Datacited by 36
- 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
- 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
- 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
- 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneitycited by 25
- 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
- 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
- 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
- 2021Exploiting Heterogeneity in Robust Federated Best-Arm Identificationcited by 20
- 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
- 2022Federated Optimization Algorithms with Random Reshuffling and Gradient Compressioncited by 18
- 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
- 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
- 2022Federated Learning on Riemannian Manifoldscited by 13
- 2023Federated Learning with Regularized Client Participationcited by 13
- 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learningcited by 13
- 2021FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learningcited by 12
- 2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Modelscited by 12
- 2024The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communicationcited by 12
- 2024Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptationcited by 12
- 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimizationcited by 11
- 2021Iterated Vector Fields and Conservatism, with Applications to Federated Learningcited by 8
- 2022A Secure Federated Learning Framework for Residential Short Term Load Forecastingcited by 7
- 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 7
- 2022Federated Learning with Server Learning: Enhancing Performance for Non-IID Datacited by 6
- 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
- 2023FilFL: Client Filtering for Optimized Client Participation in Federated Learningcited by 5
- 2023Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learningcited by 5
- 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Modelscited by 5
- 2021Accelerating Federated Learning with a Global Biased Optimisercited by 4
- 2023Faster Federated Learning with Decaying Number of Local SGD Stepscited by 4
- 2023Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labelscited by 4
- 2023Leveraging Function Space Aggregation for Federated Learning at Scalecited by 4
- 2022FedMCSA: Personalized Federated Learning via Model Components Self-Attentioncited by 3
50 most cited of 74 All 74 (JSON)
- 2026Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization27 May 2026
- 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization14 Apr 2026
- 2026High dimensional theory of two-phase optimizers27 Mar 2026
- 2026BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Training18 Jan 2026
- 2026Tight Analysis of Decentralized SGD: A Markov Chain Perspective11 Jan 2026
- 2025First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions25 Dec 2025
- 2025Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling15 Apr 2025
- 2025Distributionally Robust Federated Learning: An ADMM Algorithm24 Mar 2025
- 2025Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Data20 Mar 2025
- 2025Federated Inverse Probability Treatment Weighting for Individual Treatment Effect Estimation6 Mar 2025
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework2 Feb 2025
- 2025Regularized Top-k: A Bayesian Framework for Gradient Sparsification10 Jan 2025
38 more
- 2024Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation10 Oct 2024
- 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs27 Sep 2024
- 2024DynamicFL: Federated Learning with Dynamic Communication Resource Allocation8 Sep 2024
- 2024Neighborhood and Global Perturbations Supported SAM in Federated Learning: From Local Tweaks To Global Awareness26 Aug 2024
- 2024The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication19 May 2024
- 2024FedEGG: Federated Learning with Explicit Global Guidance18 Apr 2024
- 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models14 Mar 2024
- 2023Leveraging Function Space Aggregation for Federated Learning at Scale17 Nov 2023
- 2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models4 Oct 2023
- 2023FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data18 Sep 2023
- 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning11 Sep 2023
- 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup30 Jul 2023
- 2023Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels17 Jul 2023
- 2023FedCME: Client Matching and Classifier Exchanging to Handle Data Heterogeneity in Federated Learning17 Jul 2023
- 2023Towards a Better Theoretical Understanding of Independent Subnetwork Training28 Jun 2023
- 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization9 Jun 2023
- 2023Improving Accelerated Federated Learning with Compression and Importance Sampling5 Jun 2023
- 2023Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning22 May 2023
- 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape19 May 2023
- 2023Faster Federated Learning with Decaying Number of Local SGD Steps16 May 2023
- 2023Bayesian Federated Learning: A Survey26 Apr 2023
- 2023SalientGrads: Sparse Models for Communication Efficient and Data Aware Distributed Federated Training15 Apr 2023
- 2023Blockchain-Empowered Trustworthy Data Sharing: Fundamentals, Applications, and Challenges12 Mar 2023
- 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy21 Feb 2023
- 2023TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation20 Feb 2023
- 2023FilFL: Client Filtering for Optimized Client Participation in Federated Learning13 Feb 2023
- 2023Federated Learning with Regularized Client Participation7 Feb 2023
- 2023Federated Temporal Difference Learning with Linear Function Approximation under Environmental Heterogeneity4 Feb 2023
- 2023Federated Automatic Differentiation18 Jan 2023
- 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!29 Dec 2022
- 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimization24 Oct 2022
- 2022Communication-Efficient and Drift-Robust Federated Learning via Elastic Net6 Oct 2022
- 2022Federated Learning with Server Learning: Enhancing Performance for Non-IID Data6 Oct 2022
- 2022A Secure Federated Learning Framework for Residential Short Term Load Forecasting29 Sep 2022
- 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments24 Aug 2022
- 2022FedMCSA: Personalized Federated Learning via Model Components Self-Attention23 Aug 2022
- 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning9 Jul 2022
- 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox8 Jul 2022
50 newest of 74 All 74 (JSON)
- 2021Federated Learning Based on Dynamic Regularizationcited by 555
- 2021A Field Guide to Federated Optimizationcited by 312
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
- 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
- 2021On Large-Cohort Training for Federated Learningcited by 73
- 2021Provably Secure Federated Learning against Malicious Clientscited by 60
- 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
- 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 53
- 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
- 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
- 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
12 most cited of 74 All 74 (JSON)
- 2026Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization27 May 2026
- 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalization14 Apr 2026
- 2026High dimensional theory of two-phase optimizers27 Mar 2026
- 2026BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Training18 Jan 2026
- 2026Tight Analysis of Decentralized SGD: A Markov Chain Perspective11 Jan 2026
- 2025First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions25 Dec 2025
- 2025Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling15 Apr 2025
- 2025Distributionally Robust Federated Learning: An ADMM Algorithm24 Mar 2025
- 2025Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Data20 Mar 2025
- 2025Federated Inverse Probability Treatment Weighting for Individual Treatment Effect Estimation6 Mar 2025
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework2 Feb 2025
- 2025Regularized Top-k: A Bayesian Framework for Gradient Sparsification10 Jan 2025
12 newest of 74 All 74 (JSON)