# Avoiding Synchronization in First-Order Methods for Sparse Convex Optimization

arXiv: https://arxiv.org/abs/1712.06047 · HTML: https://mlsanity.com/paper/1712.06047/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2017.

## Built on by (most cited first, 2 of 2)

- [Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimization](https://mlsanity.com/paper/2501.07526/) (2025)
- [Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models](https://mlsanity.com/paper/2510.19999/) (2025)

## Abstract (arXiv)

Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML methods are particularly sensitive to communication cost since they often require communication every iteration. In this work, we extend well-known techniques from Communication-Avoiding Krylov subspace methods to first-order, block coordinate descent methods for Support Vector Machines and Proximal Least-Squares problems. Our Synchronization-Avoiding (SA) variants reduce the latency cost by a tunable factor of s at the expense of a factor of s increase in flops and bandwidth costs. We show that the SA-variants are numerically stable and can attain large speedups of up to 5.1× on a Cray XC30 supercomputer.