mlsanityData to 7 Oct 2026

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

arXiv 1611.02189 ↗First posted 7 Nov 2016cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 15; 2023: 11; 2024: 13; 2025: 7; 2026: 1.

69 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  3. 2022Federated Learning via Inexact ADMMcited by 44
  4. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  5. 2021Game of Gradients: Mitigating Irrelevant Clients in Federated Learningcited by 35
  6. 2021SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networkscited by 30
  7. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  8. 2021Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challengescited by 17
  9. 2024Training Neural Networks from Scratch with Parallel Low-Rank Adapterscited by 17
  10. 2021Towards Demystifying Serverless Machine Learning Trainingcited by 16
  11. 2021LocalNewton: Reducing Communication Bottleneck for Distributed Learningcited by 15
  12. 2021Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guaranteescited by 15
38 more
  1. 2021Federated Few-Shot Learning with Adversarial Learningcited by 11
  2. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
  3. 2021Joint Client Scheduling and Resource Allocation under Channel Uncertainty in Federated Learningcited by 10
  4. 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 10
  5. 2021On Second-order Optimization Methods for Federated Learningcited by 9
  6. 2023Federated Empirical Risk Minimization via Second-Order Methodcited by 9
  7. 2021Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modelingcited by 8
  8. 2022FedGiA: An Efficient Hybrid Algorithm for Federated Learningcited by 8
  9. 2022Compression and Data Similarity: Combination of Two Techniques for Communication-Efficient Solving of Distributed Variational Inequalitiescited by 8
  10. 2021Clustered Federated Learning via Generalized Total Variation Minimizationcited by 7
  11. 2025Efficient Distributed Optimization under Heavy-Tailed Noisecited by 7
  12. 2021The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communicationcited by 5
  13. 2021Linear Regression with Distributed Learning: A Generalization Error Perspectivecited by 4
  14. 2021Local and Global Uniform Convexity Conditionscited by 4
  15. 2021RingFed: Reducing Communication Costs in Federated Learning on Non-IID Datacited by 4
  16. 2022Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Settingcited by 4
  17. 2022Cerberus: Exploring Federated Prediction of Security Eventscited by 4
  18. 2022A Primal-Dual Algorithm for Hybrid Federated Learningcited by 4
  19. 2023RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learningcited by 4
  20. 2024Post-Fair Federated Learning: Achieving Group and Community Fairness in Federated Learning via Post-processingcited by 4
  21. 2023Memory-Constrained Algorithms for Convex Optimization via Recursive Cutting-Planescited by 3
  22. 2024Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradientscited by 3
  23. 2024Communication-Efficient Federated Group Distributionally Robust Optimizationcited by 3
  24. 2021Distributed Sparse Feature Selection in Communication-Restricted Networkscited by 2
  25. 2022Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computingcited by 2
  26. 2022FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differencescited by 2
  27. 2023A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learningcited by 2
  28. 2022Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topologycited by 1
  29. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  30. 2022Continual Learning with Distributed Optimization: Does CoCoA Forget?cited by 1
  31. 2023Decentralized Entropic Optimal Transport for Distributed Distribution Comparisoncited by 1
  32. 2023Distributed Semi-Supervised Sparse Statistical Inferencecited by 1
  33. 2024Optimal Data Splitting in Distributed Optimization for Machine Learningcited by 1
  34. 2024Analysis of Total Variation Minimization for Clustered Federated Learningcited by 1
  35. 2024Gradient Descent is Pareto-Optimal in the Oracle Complexity and Memory Tradeoff for Feasibility Problemscited by 1
  36. 2024Communication-Efficient Distributed Deep Learning via Federated Dynamic Averagingcited by 1
  37. 2024Local Methods with Adaptivity via Scalingcited by 1
  38. 2024Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classificationcited by 1

50 most cited of 69 All 69 (JSON)