# CoCoA: A General Framework for Communication-Efficient Distributed Optimization

arXiv: https://arxiv.org/abs/1611.02189 · HTML: https://mlsanity.com/paper/1611.02189/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2016.

## Built on by (most cited first, 30 of 69)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [FedMix: Approximation of Mixup under Mean Augmented Federated Learning](https://mlsanity.com/paper/2107.00233/) (2021)
- [Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2204.10607/) (2022)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [Game of Gradients: Mitigating Irrelevant Clients in Federated Learning](https://mlsanity.com/paper/2110.12257/) (2021)
- [SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks](https://mlsanity.com/paper/2106.02743/) (2021)
- [FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity](https://mlsanity.com/paper/2204.03529/) (2022)
- [Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges](https://mlsanity.com/paper/2102.01375/) (2021)
- [Training Neural Networks from Scratch with Parallel Low-Rank Adapters](https://mlsanity.com/paper/2402.16828/) (2024)
- [Towards Demystifying Serverless Machine Learning Training](https://mlsanity.com/paper/2105.07806/) (2021)
- [LocalNewton: Reducing Communication Bottleneck for Distributed Learning](https://mlsanity.com/paper/2105.07320/) (2021)
- [Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees](https://mlsanity.com/paper/2110.03313/) (2021)
- [Federated Few-Shot Learning with Adversarial Learning](https://mlsanity.com/paper/2104.00365/) (2021)
- [Over-the-Air Federated Learning via Second-Order Optimization](https://mlsanity.com/paper/2203.15488/) (2022)
- [FeDXL: Provable Federated Learning for Deep X-Risk Optimization](https://mlsanity.com/paper/2210.14396/) (2022)
- [Joint Client Scheduling and Resource Allocation under Channel Uncertainty in Federated Learning](https://mlsanity.com/paper/2106.06796/) (2021)
- [On Second-order Optimization Methods for Federated Learning](https://mlsanity.com/paper/2109.02388/) (2021)
- [Federated Empirical Risk Minimization via Second-Order Method](https://mlsanity.com/paper/2305.17482/) (2023)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [FedGiA: An Efficient Hybrid Algorithm for Federated Learning](https://mlsanity.com/paper/2205.01438/) (2022)
- [Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalities](https://mlsanity.com/paper/2206.09446/) (2022)
- [Clustered Federated Learning via Generalized Total Variation Minimization](https://mlsanity.com/paper/2105.12769/) (2021)
- [Efficient Distributed Optimization under Heavy-Tailed Noise](https://mlsanity.com/paper/2502.04164/) (2025)
- [The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication](https://mlsanity.com/paper/2103.13026/) (2021)
- [Linear Regression with Distributed Learning: A Generalization Error Perspective](https://mlsanity.com/paper/2101.09001/) (2021)
- [Local and Global Uniform Convexity Conditions](https://mlsanity.com/paper/2102.05134/) (2021)
- [RingFed: Reducing Communication Costs in Federated Learning on Non-IID Data](https://mlsanity.com/paper/2107.08873/) (2021)
- [Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Setting](https://mlsanity.com/paper/2202.02580/) (2022)
- [Cerberus: Exploring Federated Prediction of Security Events](https://mlsanity.com/paper/2209.03050/) (2022)
- [A Primal-Dual Algorithm for Hybrid Federated Learning](https://mlsanity.com/paper/2210.08106/) (2022)

## Abstract (arXiv)

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets.