mlsanityData to 14 Oct 2026

Global Convergence of ADMM in Nonconvex Nonsmooth Optimization

arXiv 1511.06324 ↗First posted 18 Nov 2015math.OC cs.NA math.NAcited by 98

Not on the map. Why?

Abstract (arXiv)

In this paper, we analyze the convergence of the alternating direction method of multipliers (ADMM) for minimizing a nonconvex and possibly nonsmooth objective function, φ(x0,ldots,xp,y), subject to coupled linear equality constraints. Our ADMM updates each of the primal variables x0,ldots,xp,y, followed by updating the dual variable. We separate the variable y from xi's as it has a special role in our analysis. The developed convergence guarantee covers a variety of nonconvex functions such as piecewise linear functions, ℓq quasi-norm, Schatten-q quasi-norm (0<q<1), minimax concave penalty (MCP), and smoothly clipped absolute deviation (SCAD) penalty. It also allows nonconvex constraints such as compact manifolds (e.g., spherical, Stiefel, and Grassman manifolds) and linear complementarity constraints. Also, the x0-block can be almost any lower semi-continuous function. By applying our analysis, we show, for the first time, that several ADMM algorithms applied to solve nonconvex models in statistical learning, optimization on manifold, and matrix decomposition are guaranteed to converge. Our results provide sufficient conditions for ADMM to converge on (convex or nonconvex) monotropic programs with three or more blocks, as they are special cases of our model. ADMM has been regarded as a variant to the augmented Lagrangian method (ALM). We present a simple example to illustrate how ADMM converges but ALM diverges with bounded penalty parameter β. Indicated by this example and other analysis in this paper, ADMM might be a better choice than ALM for some nonconvex nonsmooth problems, because ADMM is not only easier to implement, it is also more likely to converge for the concerned scenarios.

Key facts. Cited by 98 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 19; 2022: 10; 2023: 21; 2024: 13; 2025: 17; 2026: 17.

98 papers.

Sort:

By area:

  1. 2020Imitation Learning with Stability and Safety Guaranteescited by 34
  2. 2023Learning to Warm-Start Fixed-Point Optimization Algorithmscited by 32
  3. 2021Towards Federated Bayesian Network Structure Learning with Continuous Optimizationcited by 31
  4. 2024ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Modelscited by 19
  5. 2021Convergence of Random Reshuffling Under The Kurdyka-Lojasiewicz Inequalitycited by 16
  6. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16
  7. 2021Nonlinear Transform Induced Tensor Nuclear Norm for Tensor Completioncited by 13
  8. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  9. 2022Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patternscited by 11
  10. 2022Joint Network Topology Inference via a Shared Graphon Modelcited by 11
  11. 2023Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimizationcited by 11
  12. 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Driftcited by 10
38 more
  1. 2023Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimizationcited by 9
  2. 2021Training Recurrent Neural Networks by Sequential Least Squares and the Alternating Direction Method of Multiplierscited by 8
  3. 2023vSHARP: variable Splitting Half-quadratic Admm algorithm for Reconstruction of inverse-Problemscited by 8
  4. 2021A unifying framework for n-dimensional quasi-conformal mappingscited by 7
  5. 2023Enhanced Latent Multi-view Subspace Clusteringcited by 6
  6. 2025AugFL: Augmenting Federated Learning with Pretrained Modelscited by 6
  7. 2025LensNet: An End-to-End Learning Framework for Empirical Point Spread Function Modeling and Lensless Imaging Reconstructioncited by 6
  8. 2021LADMM-Net: An Unrolled Deep Network For Spectral Image Fusion From Compressive Datacited by 5
  9. 2022Multi-channel Nuclear Norm Minus Frobenius Norm Minimization for Color Image Denoisingcited by 5
  10. 2023Interaction-Aware Trajectory Planning for Autonomous Vehicles with Analytic Integration of Neural Networks into Model Predictive Controlcited by 5
  11. 2024Towards Certification of Uncertainty Calibration under Adversarial Attackscited by 5
  12. 2025SAFE: Finding Sparse and Flat Minima to Improve Pruningcited by 5
  13. 2022Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Settingcited by 4
  14. 2023Smoothing ADMM for Sparse-Penalized Quantile Regression with Non-Convex Penaltiescited by 4
  15. 2021Minimizing L1 over L2 norms on the gradientcited by 3
  16. 2021A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimizationcited by 3
  17. 2022Hierarchical fuzzy neural networks with privacy preservation for heterogeneous big datacited by 3
  18. 2022A Riemannian ADMMcited by 3
  19. 2023Weighted Anisotropic-Isotropic Total Variation for Poisson Denoisingcited by 3
  20. 2023Robust Feature Inference: A Test-time Defense Strategy using Spectral Projectionscited by 3
  21. 2024Multiview Graph Learning with Consensus Graphcited by 3
  22. 2024Practical Marketplace Optimization at Uber Using Causally-Informed Machine Learningcited by 3
  23. 2025Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositionscited by 3
  24. 2021Stochastic Block-ADMM for Training Deep Networkscited by 2
  25. 2021Multi-Tensor Network Representation for High-Order Tensor Completioncited by 2
  26. 2023RED-PSM: Regularization by Denoising of Factorized Low Rank Models for Dynamic Imagingcited by 2
  27. 2024Privacy-Preserving Distributed Nonnegative Matrix Factorizationcited by 2
  28. 2024Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penaltiescited by 2
  29. 2024HR-Bandit: Human-AI Collaborated Linear Recourse Banditcited by 2
  30. 2024Low-Rank Tensors for Multi-Dimensional Markov Modelscited by 2
  31. 2025Adaptive Pruning of Pretrained Transformer via Differential Inclusionscited by 2
  32. 2026Dual-space posterior sampling for Bayesian inference in constrained inverse problemscited by 2
  33. 2026Learning Multi-Agent Coordination via Sheaf-ADMMcited by 2
  34. 2021Hierarchical Non-Stationary Temporal Gaussian Processes With L1-Regularizationcited by 1
  35. 2021Adaptively Weighted Top-N Recommendation for Organ Matchingcited by 1
  36. 2021A Convergent ADMM Framework for Efficient Neural Network Trainingcited by 1
  37. 2022Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularizationcited by 1
  38. 2024Deep Few-view High-resolution Photon-counting CT at Halved Dose for Extremity Imagingcited by 1

50 most cited of 98 All 98 (JSON)