# On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization

arXiv: https://arxiv.org/abs/2502.00470 · HTML: https://mlsanity.com/paper/2502.00470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Builds on (most cited first, 12 of 12)

- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2204.10607/) (2022)
- [Communication-Efficient Distributed Dual Coordinate Ascent](https://mlsanity.com/paper/1409.1458/) (2014)
- [COLA: Decentralized Linear Learning](https://mlsanity.com/paper/1808.04883/) (2018)
- [Adding vs. Averaging in Distributed Primal-Dual Optimization](https://mlsanity.com/paper/1502.03508/) (2015)
- [Parallel Multi-Block ADMM with o(1/k) Convergence](https://mlsanity.com/paper/1312.3040/) (2013)
- [A Distributed Second-Order Algorithm You Can Trust](https://mlsanity.com/paper/1806.07569/) (2018)
- [A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features](https://mlsanity.com/paper/2208.03700/) (2022)
- [On a Unified and Simplified Proof for the Ergodic Convergence Rates of PPM, PDHG and ADMM](https://mlsanity.com/paper/2305.02165/) (2023)
- [L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework](https://mlsanity.com/paper/1512.04011/) (2015)
- [Distributed Block-diagonal Approximation Methods for Regularized Empirical Risk Minimization](https://mlsanity.com/paper/1709.03043/) (2017)
- [An Accelerated Communication-Efficient Primal-Dual Optimization Framework for Structured Machine Learning](https://mlsanity.com/paper/1711.05305/) (2017)

## Abstract (arXiv)

Distributed empirical risk minimization (ERM) is often studied through two influential yet seemingly separate families of methods: CoCoA-type algorithms, derived from distributed dual coordinate ascent, and ADMM-type algorithms, derived from consensus and proximal splitting. In this paper, we investigate the connection of the two types of algorithms from a unified primal-dual perspective. We show that consensus ADMM, linearized consensus ADMM, two distributed proximal ADMM variants, and ridge-regularized CoCoA can all be written in a common update form involving a global primal variable and block dual variables. This reformulation makes several previously hidden connections explicit: For ridge-regularized ERM, CoCoA coincides with a particular proximal ADMM scheme at the level of the dual update. Moreover, consensus ADMM on the primal problem is equivalent to proximal ADMM on the dual problem under an explicit parameter mapping together with a sign reversal of the saddle objective; similar correspondences also hold for the linearized variants. These results indicates that the ADMM-type algorithms, when fine tuned, performs at least as good as CoCoA, under ridge regularized ERM problems. The unified view also yields a natural primal-dual gap stopping criterion for consensus ADMM and a unified O(1/T) ergodic convergence analysis for the ADMM-type methods. Experiments on synthetic regression problems and real SVM datasets support the predicted relationships, clarify the role of tuning parameters, and show that suitably tuned ADMM variants can outperform CoCoA in the ridge-regularized setting.