# Accelerated Variance Reduced Stochastic ADMM

arXiv: https://arxiv.org/abs/1707.03190 · HTML: https://mlsanity.com/paper/1707.03190/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2017.

## Built on by (most cited first, 5 of 5)

- [Preconditioned Inexact Stochastic ADMM for Deep Model](https://mlsanity.com/paper/2502.10784/) (2025)
- [Robust Regression via Model Based Methods](https://mlsanity.com/paper/2106.10759/) (2021)
- [Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT](https://mlsanity.com/paper/2107.00481/) (2021)
- [Provable Constrained Stochastic Convex Optimization with XOR-Projected Gradient Descent](https://mlsanity.com/paper/2203.11829/) (2022)
- [A General Continuous-Time Formulation of Stochastic ADMM and Its Variants](https://mlsanity.com/paper/2404.14358/) (2024)

## Abstract (arXiv)

Recently, many variance reduced stochastic alternating direction method of multipliers (ADMM) methods (e.g. SAG-ADMM, SDCA-ADMM and SVRG-ADMM) have made exciting progress such as linear convergence rates for strongly convex problems. However, the best known convergence rate for general convex problems is O(1/T) as opposed to O(1/T2) of accelerated batch algorithms, where T is the number of iterations. Thus, there still remains a gap in convergence rates between existing stochastic ADMM and batch algorithms. To bridge this gap, we introduce the momentum acceleration trick for batch optimization into the stochastic variance reduced gradient based ADMM (SVRG-ADMM), which leads to an accelerated (ASVRG-ADMM) method. Then we design two different momentum term update rules for strongly convex and general convex cases. We prove that ASVRG-ADMM converges linearly for strongly convex problems. Besides having a low per-iteration complexity as existing stochastic ADMM methods, ASVRG-ADMM improves the convergence rate on general convex problems from O(1/T) to O(1/T2). Our experimental results show the effectiveness of ASVRG-ADMM.