# Make Workers Work Harder: Decoupled Asynchronous Proximal Stochastic Gradient Descent

arXiv: https://arxiv.org/abs/1605.06619 · HTML: https://mlsanity.com/paper/1605.06619/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2016.

## Built on by (most cited first, 1 of 1)

- [Distributed Dynamic Safe Screening Algorithms for Sparse Regularization](https://mlsanity.com/paper/2204.10981/) (2022)

## Abstract (arXiv)

Asynchronous parallel optimization algorithms for solving large-scale machine learning problems have drawn significant attention from academia to industry recently. This paper proposes a novel algorithm, decoupled asynchronous proximal stochastic gradient descent (DAP-SGD), to minimize an objective function that is the composite of the average of multiple empirical losses and a regularization term. Unlike the traditional asynchronous proximal stochastic gradient descent (TAP-SGD) in which the master carries much of the computation load, the proposed algorithm off-loads the majority of computation tasks from the master to workers, and leaves the master to conduct simple addition operations. This strategy yields an easy-to-parallelize algorithm, whose performance is justified by theoretical convergence analyses. To be specific, DAP-SGD achieves an O(log T/T) rate when the step-size is diminishing and an ergodic O(1/√T) rate when the step-size is constant, where T is the number of total iterations.