# Avoiding Communication in Logistic Regression

arXiv: https://arxiv.org/abs/2011.08281 · HTML: https://mlsanity.com/paper/2011.08281/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Communication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimization](https://mlsanity.com/paper/2501.07526/) (2025)
- [Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models](https://mlsanity.com/paper/2510.19999/) (2025)
- [Mixed-Precision Communication-Avoiding SGD for Generalized Linear Models on GPUs](https://mlsanity.com/paper/2606.18463/) (2026)

## Abstract (arXiv)

Stochastic gradient descent (SGD) is one of the most widely used optimization methods for solving various machine learning problems. SGD solves an optimization problem by iteratively sampling a few data points from the input data, computing gradients for the selected data points, and updating the solution. However, in a parallel setting, SGD requires interprocess communication at every iteration. We introduce a new communication-avoiding technique for solving the logistic regression problem using SGD. This technique re-organizes the SGD computations into a form that communicates every s iterations instead of every iteration, where s is a tuning parameter. We prove theoretical flops, bandwidth, and latency upper bounds for SGD and its new communication-avoiding variant. Furthermore, we show experimental results that illustrate that the new Communication-Avoiding SGD (CA-SGD) method can achieve speedups of up to 4.97× on a high-performance Infiniband cluster without altering the convergence behavior or accuracy.