# BALPA: A Balanced Primal-Dual Algorithm for Nonsmooth Optimization with Application to Distributed Optimization

arXiv: https://arxiv.org/abs/2212.02835 · HTML: https://mlsanity.com/paper/2212.02835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent](https://mlsanity.com/paper/1905.11261/) (2019)
- [Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop](https://mlsanity.com/paper/1901.08689/) (2019)
- [Variance Reduced Stochastic Gradient Descent with Neighbors](https://mlsanity.com/paper/1506.03662/) (2015)
- [A Three-Operator Splitting Scheme and its Optimization Applications](https://mlsanity.com/paper/1504.01032/) (2015)
- [A New Randomized Block-Coordinate Primal-Dual Proximal Algorithm for Distributed Optimization](https://mlsanity.com/paper/1706.02882/) (2017)
- [A new primal-dual algorithm for minimizing the sum of three functions with a linear operator](https://mlsanity.com/paper/1611.09805/) (2016)
- [Asymmetric Forward-Backward-Adjoint Splitting for Solving Monotone Inclusions Involving Three Operators](https://mlsanity.com/paper/1602.08729/) (2016)
- [Accelerated first-order primal-dual proximal methods for linearly constrained composite convex programming](https://mlsanity.com/paper/1606.09155/) (2016)
- [Acceleration of Primal-Dual Methods by Preconditioning and Simple Subproblem Procedures](https://mlsanity.com/paper/1811.08937/) (2018)
- [On the Equivalence of Inexact Proximal ALM and ADMM for a Class of Convex Composite Programming](https://mlsanity.com/paper/1803.10803/) (2018)
- [Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms](https://mlsanity.com/paper/2004.02635/) (2020)
- [DISA: A Dual Inexact Splitting Algorithm for Distributed Convex Composite Optimization](https://mlsanity.com/paper/2209.01850/) (2022)
- [A primal-dual fixed-point algorithm for minimization of the sum of three convex separable functions](https://mlsanity.com/paper/1512.09235/) (2015)
- [Asynchronous Distributed Voltage Control in Active Distribution Networks](https://mlsanity.com/paper/1908.06548/) (2019)
- [Balanced Augmented Lagrangian Method for Convex Programming](https://mlsanity.com/paper/2108.08554/) (2021)

## Abstract (arXiv)

In this paper, we propose a novel primal-dual proximal splitting algorithm (PD-PSA), named BALPA, for the composite optimization problem with equality constraints, where the loss function consists of a smooth term and a nonsmooth term composed with a linear mapping. In BALPA, the dual update is designed as a proximal point for a time-varying quadratic function, which balances the implementation of primal and dual update and retains the proximity-induced feature of classic PD-PSAs. In addition, by this balance, BALPA eliminates the inefficiency of classic PD-PSAs for composite optimization problems in which the Euclidean norm of the linear mapping or the equality constraint mapping is large. Therefore, BALPA not only inherits the advantages of simple structure and easy implementation of classic PD-PSAs but also ensures a fast convergence when these norms are large. Moreover, we propose a stochastic version of BALPA (S-BALPA) and apply the developed BALPA to distributed optimization to devise a new distributed optimization algorithm. Furthermore, a comprehensive convergence analysis for BALPA and S-BALPA is conducted, respectively. Finally, numerical experiments demonstrate the efficiency of the proposed algorithms.