# Adaptive Stochastic Alternating Direction Method of Multipliers

arXiv: https://arxiv.org/abs/1312.4564 · HTML: https://mlsanity.com/paper/1312.4564/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2013.

## Built on by (most cited first, 1 of 1)

- [Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers](https://mlsanity.com/paper/2411.06135/) (2024)

## Abstract (arXiv)

The Alternating Direction Method of Multipliers (ADMM) has been studied for years. The traditional ADMM algorithm needs to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a computational complexity proportional to the number of training examples. To reduce the time complexity, stochastic ADMM algorithms were proposed to replace the expected function with a random loss function associated with one uniformly drawn example plus a Bregman divergence. The Bregman divergence, however, is derived from a simple second order proximal function, the half squared norm, which could be a suboptimal choice. In this paper, we present a new family of stochastic ADMM algorithms with optimal second order proximal functions, which produce a new family of adaptive subgradient methods. We theoretically prove that their regret bounds are as good as the bounds which could be achieved by the best proximal function that can be chosen in hindsight. Encouraging empirical results on a variety of real-world datasets confirm the effectiveness and efficiency of the proposed algorithms.