# Communication-Efficient Distributed Dual Coordinate Ascent

arXiv: https://arxiv.org/abs/1409.1458 · HTML: https://mlsanity.com/paper/1409.1458/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2014.

## Built on by (most cited first, 30 of 31)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems](https://mlsanity.com/paper/2101.10761/) (2021)
- [SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks](https://mlsanity.com/paper/2106.02743/) (2021)
- [Can 5th Generation Local Training Methods Support Client Sampling? Yes!](https://mlsanity.com/paper/2212.14370/) (2022)
- [Efficient Convex Optimization Requires Superlinear Memory](https://mlsanity.com/paper/2203.15260/) (2022)
- [FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity](https://mlsanity.com/paper/2404.09816/) (2024)
- [Federated Empirical Risk Minimization via Second-Order Method](https://mlsanity.com/paper/2305.17482/) (2023)
- [Communication-Efficient Distributed Optimization with Quantized Preconditioners](https://mlsanity.com/paper/2102.07214/) (2021)
- [Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms](https://mlsanity.com/paper/2310.15330/) (2023)
- [Personalized Federated Learning with Communication Compression](https://mlsanity.com/paper/2209.05148/) (2022)
- [Linear Regression with Distributed Learning: A Generalization Error Perspective](https://mlsanity.com/paper/2101.09001/) (2021)
- [Local and Global Uniform Convexity Conditions](https://mlsanity.com/paper/2102.05134/) (2021)
- [A Primal-Dual Algorithm for Hybrid Federated Learning](https://mlsanity.com/paper/2210.08106/) (2022)
- [Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation](https://mlsanity.com/paper/2301.03125/) (2023)
- [FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning](https://mlsanity.com/paper/2303.05206/) (2023)
- [RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learning](https://mlsanity.com/paper/2306.00675/) (2023)
- [Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients](https://mlsanity.com/paper/2406.01439/) (2024)
- [Scalable Dual Coordinate Descent for Kernel Methods](https://mlsanity.com/paper/2406.18001/) (2024)
- [Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applications](https://mlsanity.com/paper/2101.01505/) (2021)
- [FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences](https://mlsanity.com/paper/2210.09626/) (2022)
- [Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topology](https://mlsanity.com/paper/2207.09657/) (2022)
- [Continual Learning with Distributed Optimization: Does CoCoA Forget?](https://mlsanity.com/paper/2211.16994/) (2022)
- [Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classification](https://mlsanity.com/paper/2406.01753/) (2024)
- [L-XAIDS: A LIME-based eXplainable AI framework for Intrusion Detection Systems](https://mlsanity.com/paper/2508.17244/) (2025)
- [On the Optimal Batch Size for Byzantine-Robust Distributed Learning](https://mlsanity.com/paper/2305.13856/) (2023)
- [Distributed Dual Coordinate Ascent with Imbalanced Data on a General Tree Network](https://mlsanity.com/paper/2308.14783/) (2023)
- [Personalized Federated Learning via ADMM with Moreau Envelope](https://mlsanity.com/paper/2311.06756/) (2023)
- [Distributed Continual Learning with CoCoA in High-dimensional Linear Regression](https://mlsanity.com/paper/2312.01795/) (2023)
- [FedScalar: Federated Learning with Scalar Communication for Bandwidth-Constrained Networks](https://mlsanity.com/paper/2410.02260/) (2024)
- [On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization](https://mlsanity.com/paper/2502.00470/) (2025)

## Abstract (arXiv)

Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that uses local computation in a primal-dual setting to dramatically reduce the amount of necessary communication. We provide a strong convergence rate analysis for this class of algorithms, as well as experiments on real-world distributed datasets with implementations in Spark. In our experiments, we find that as compared to state-of-the-art mini-batch versions of SGD and SDCA algorithms, CoCoA converges to the same .001-accurate solution quality on average 25x as quickly.