# Analysis of Generalized Bregman Surrogate Algorithms for Nonsmooth Nonconvex Statistical Learning

arXiv: https://arxiv.org/abs/2112.09191 · HTML: https://mlsanity.com/paper/2112.09191/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 67 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 6 of 6)

- [Accelerated Gradient Methods for Nonconvex Nonlinear and Stochastic Programming](https://mlsanity.com/paper/1310.3787/) (2013)
- [Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima](https://mlsanity.com/paper/1305.2436/) (2013)
- [Optimal computational and statistical rates of convergence for sparse nonconvex learning problems](https://mlsanity.com/paper/1306.4960/) (2013)
- [Robust Reduced Rank Regression](https://mlsanity.com/paper/1509.03938/) (2015)
- [Gaining Outlier Resistance with Progressive Quantiles: Fast Algorithms and Theoretical Studies](https://mlsanity.com/paper/2112.08471/) (2021)
- [Learning Topology and Dynamics of Large Recurrent Neural Networks](https://mlsanity.com/paper/1410.1174/) (2014)

## Built on by (most cited first, 5 of 5)

- [Foundations of Top-k Decoding For Language Models](https://mlsanity.com/paper/2505.19371/) (2025)
- [Supervised Multivariate Learning with Simultaneous Feature Auto-grouping and Dimension Reduction](https://mlsanity.com/paper/2112.09746/) (2021)
- [Robust Regularized Low-Rank Matrix Models for Regression and Classification](https://mlsanity.com/paper/2205.07106/) (2022)
- [Slow Kill for Big Data Learning](https://mlsanity.com/paper/2305.01726/) (2023)
- [Range Penalization: Theoretical Insights with Applications in Federated Learning](https://mlsanity.com/paper/2606.10916/) (2026)

## Abstract (arXiv)

Modern statistical applications often involve minimizing an objective function that may be nonsmooth and/or nonconvex. This paper focuses on a broad Bregman-surrogate algorithm framework including the local linear approximation, mirror descent, iterative thresholding, DC programming and many others as particular instances. The recharacterization via generalized Bregman functions enables us to construct suitable error measures and establish global convergence rates for nonconvex and nonsmooth objectives in possibly high dimensions. For sparse learning problems with a composite objective, under some regularity conditions, the obtained estimators as the surrogate's fixed points, though not necessarily local minimizers, enjoy provable statistical guarantees, and the sequence of iterates can be shown to approach the statistical truth within the desired accuracy geometrically fast. The paper also studies how to design adaptive momentum based accelerations without assuming convexity or smoothness by carefully controlling stepsize and relaxation parameters.