# Proximal Algorithms in Statistics and Machine Learning

arXiv: https://arxiv.org/abs/1502.03175 · HTML: https://mlsanity.com/paper/1502.03175/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2015.

## Built on by (most cited first, 18 of 18)

- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [Denoising: A Powerful Building-Block for Imaging, Inverse Problems, and Machine Learning](https://mlsanity.com/paper/2409.06219/) (2024)
- [Faster Deep Reinforcement Learning with Slower Online Network](https://mlsanity.com/paper/2112.05848/) (2021)
- [Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs](https://mlsanity.com/paper/2204.08621/) (2022)
- [Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics](https://mlsanity.com/paper/2309.04522/) (2023)
- [Bayesian Learning of Parameterised Quantum Circuits](https://mlsanity.com/paper/2206.07559/) (2022)
- [Sparse Bayesian Lasso via a Variable-Coefficient ℓ1 Penalty](https://mlsanity.com/paper/2211.05089/) (2022)
- [POLA: Online Time Series Prediction by Adaptive Learning Rates](https://mlsanity.com/paper/2102.08907/) (2021)
- [Robust online joint state/input/parameter estimation of linear systems](https://mlsanity.com/paper/2204.05663/) (2022)
- [A Probabilistic Basis for Low-Rank Matrix Learning](https://mlsanity.com/paper/2510.05447/) (2025)
- [Active-set algorithms based statistical inference for shape-restricted generalized additive Cox regression models](https://mlsanity.com/paper/2106.15735/) (2021)
- [Bayesian Error-in-Variables Models for the Identification of Power Networks](https://mlsanity.com/paper/2107.04480/) (2021)
- [Accelerated nonlinear primal-dual hybrid gradient methods with applications to supervised machine learning](https://mlsanity.com/paper/2109.12222/) (2021)
- [Riesz-Quincunx-UNet Variational Auto-Encoder for Satellite Image Denoising](https://mlsanity.com/paper/2208.12810/) (2022)
- [Approximation of the Proximal Operator of the ℓ_∞ Norm Using a Neural Network](https://mlsanity.com/paper/2408.11211/) (2024)
- [Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression](https://mlsanity.com/paper/2505.11143/) (2025)
- [Beyond sparse denoising in frames: minimax estimation with a scattering transform](https://mlsanity.com/paper/2510.19612/) (2025)
- [Structure Learning on Clustered Data](https://mlsanity.com/paper/2607.08238/) (2026)

## Abstract (arXiv)

In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Forward-Backward, Douglas-Rachford and Half-Quadratic envelopes. Envelope representations lead to novel proximal algorithms for statistical optimisation of composite objective functions which include both non-smooth and non-convex objectives. We illustrate our methodology with regularized Logistic and Poisson regression and non-convex bridge penalties with a fused lasso norm. We provide a discussion of convergence of non-descent algorithms with acceleration and for non-convex functions. Finally, we provide directions for future research.