mlsanityData to 7 Oct 2026

Distributed Optimization with Arbitrary Local Solvers

arXiv 1512.04039 ↗First posted 13 Dec 2015cs.LG math.OCcited by 23

Not on the map. Why?

Abstract (arXiv)

With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Recent work in this area has been limited by focusing heavily on developing highly specific methods for the distributed environment. These special-purpose methods are often unable to fully leverage the competitive performance of their well-tuned and customized single machine counterparts. Further, they are unable to easily integrate improvements that continue to be made to single machine methods. To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally, and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods. We give strong primal-dual convergence rate guarantees for our framework that hold for arbitrary local solvers. We demonstrate the impact of local solver selection both theoretically and in an extensive experimental comparison. Finally, we provide thorough implementation details for our framework, highlighting areas for practical performance gains.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 7; 2023: 6; 2024: 0; 2025: 2; 2026: 1.

23 papers.

Sort:

  1. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  2. 2021Distributed Second Order Methods with Fast Rates and Compressed Communicationcited by 25
  3. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  4. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  5. 2021Asynchronous Federated Learning for Sensor Data with Concept Driftcited by 14
  6. 2023Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devicescited by 13
  7. 2021Communication-Efficient Hierarchical Federated Learning for IoT Heterogeneous Systems with Imbalanced Datacited by 10
  8. 2021Distributed Conditional Generative Adversarial Networks (GANs) for Data-Driven Millimeter Wave Communications in UAV Networkscited by 8
  9. 2022Adaptive Learning Rates for Faster Stochastic Gradient Methodscited by 7
  10. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  11. 2021Linear Regression with Distributed Learning: A Generalization Error Perspectivecited by 4
  12. 2023Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimizationcited by 3
11 more
  1. 2022FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differencescited by 2
  2. 2022Optimal Privacy Preserving for Federated Learning in Mobile Edge Computingcited by 1
  3. 2022Continual Learning with Distributed Optimization: Does CoCoA Forget?cited by 1
  4. 2023Learner Referral for Cost-Effective Federated Learning Over Hierarchical IoT Networkscited by 1
  5. 2025Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimizationcited by 1
  6. 2021Optimization of User Selection and Bandwidth Allocation for Federated Learning in VLC/RF Systemscited by 0
  7. 2021Federated Learning with Superquantile Aggregation for Heterogeneous Datacited by 0
  8. 2023Client Orchestration and Cost-Efficient Joint Optimization for NOMA-Enabled Hierarchical Federated Learningcited by 0
  9. 2023Distributed Continual Learning with CoCoA in High-dimensional Linear Regressioncited by 0
  10. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0
  11. 2026Byzantine-Robust Optimization under (L0, L1)-Smoothnesscited by 0