# Gradient Sampling Methods for Nonsmooth Optimization

arXiv: https://arxiv.org/abs/1804.11003 · HTML: https://mlsanity.com/paper/1804.11003/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2018.

## Built on by (most cited first, 23 of 23)

- [Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2209.05045/) (2022)
- [Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints](https://mlsanity.com/paper/2402.18012/) (2024)
- [Deterministic Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2302.08300/) (2023)
- [Efficient Gradient Approximation Method for Constrained Bilevel Optimization](https://mlsanity.com/paper/2302.01970/) (2023)
- [Training Recurrent Neural Networks by Sequential Least Squares and the Alternating Direction Method of Multipliers](https://mlsanity.com/paper/2112.15348/) (2021)
- [An L-BFGS-B approach for linear and nonlinear system identification under ℓ1 and group-Lasso regularization](https://mlsanity.com/paper/2403.03827/) (2024)
- [Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferential](https://mlsanity.com/paper/2210.11577/) (2022)
- [Smoothing Methods for Automatic Differentiation Across Conditional Branches](https://mlsanity.com/paper/2310.03585/) (2023)
- [Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies](https://mlsanity.com/paper/2210.04810/) (2022)
- [Optimization and Optimizers for Adversarial Robustness](https://mlsanity.com/paper/2303.13401/) (2023)
- [Model-Free μ Synthesis via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2111.15537/) (2021)
- [D-GRIL: End-to-End Topological Learning with 2-parameter Persistence](https://mlsanity.com/paper/2406.07100/) (2024)
- [Survey Descent: A Multipoint Generalization of Gradient Descent for Nonsmooth Optimization](https://mlsanity.com/paper/2111.15645/) (2021)
- [Optimization for Robustness Evaluation beyond ℓp Metrics](https://mlsanity.com/paper/2210.00621/) (2022)
- [Stochastic Gradient Sampling for Enhancing Neural Networks Training](https://mlsanity.com/paper/2310.17042/) (2023)
- [Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection](https://mlsanity.com/paper/2407.18707/) (2024)
- [The Normalized Maximum Likelihood for Regular Non-Smooth Models: Measure-Theoretic Foundations and Geometric Sampling](https://mlsanity.com/paper/2605.24477/) (2026)
- [Markov subsampling based Huber Criterion](https://mlsanity.com/paper/2112.06134/) (2021)
- [Model-Free μ-Synthesis: A Nonsmooth Optimization Perspective](https://mlsanity.com/paper/2402.11654/) (2024)
- [SPGD: Steepest Perturbed Gradient Descent Optimization](https://mlsanity.com/paper/2411.04946/) (2024)
- [Permutation Randomization on Nonsmooth Nonconvex Optimization: A Theoretical and Experimental Study](https://mlsanity.com/paper/2505.11752/) (2025)
- [Lipschitzian SLLNs for random functions](https://mlsanity.com/paper/2607.20411/) (2026)
- [The Exponential Price of Determinism in Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2609.23837/) (2026)

## Abstract (arXiv)

This paper reviews the gradient sampling methodology for solving nonsmooth, nonconvex optimization problems. An intuitively straightforward gradient sampling algorithm is stated and its convergence properties are summarized. Throughout this discussion, we emphasize the simplicity of gradient sampling as an extension of the steepest descent method for minimizing smooth objectives. We then provide overviews of various enhancements that have been proposed to improve practical performance, as well as of several extensions that have been made in the literature, such as to solve constrained problems. The paper also includes clarification of certain technical aspects of the analysis of gradient sampling algorithms, most notably related to the assumptions one needs to make about the set of points at which the objective is continuously differentiable. Finally, we discuss possible future research directions.