# Langevin dynamics based algorithm e-THvarepsilonO POULA for stochastic optimization problems with discontinuous stochastic gradient

arXiv: https://arxiv.org/abs/2210.13193 · HTML: https://mlsanity.com/paper/2210.13193/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 81 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [Analysis of Langevin Monte Carlo via convex optimization](https://mlsanity.com/paper/1802.09188/) (2018)
- [User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient](https://mlsanity.com/paper/1710.00095/) (2017)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization](https://mlsanity.com/paper/1707.06618/) (2017)
- [Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau](https://mlsanity.com/paper/1612.07471/) (2016)
- [Sharp convergence rates for Langevin dynamics in the nonconvex setting](https://mlsanity.com/paper/1805.01648/) (2018)
- [The promises and pitfalls of Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/1811.10072/) (2018)
- [Approximation Bounds for Random Neural Networks and Reservoir Systems](https://mlsanity.com/paper/2002.05933/) (2020)
- [Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization](https://mlsanity.com/paper/1910.02008/) (2019)
- [The Tamed Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1710.05559/) (2017)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [Taming neural networks with TUSLA: Non-convex learning via adaptive stochastic gradient Langevin algorithms](https://mlsanity.com/paper/2006.14514/) (2020)
- [On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case](https://mlsanity.com/paper/1812.02709/) (2018)
- [Deep Neural Network Framework Based on Backward Stochastic Differential Equations for Pricing and Hedging American Options in High Dimensions](https://mlsanity.com/paper/1909.11532/) (2019)
- [Insurance Premium Prediction via Gradient Tree-Boosted Tweedie Compound Poisson Models](https://mlsanity.com/paper/1508.06378/) (2015)
- [Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks](https://mlsanity.com/paper/2105.13937/) (2021)
- [Convergence of a Stochastic Subgradient Method with Averaging for Nonsmooth Nonconvex Constrained Optimization](https://mlsanity.com/paper/1912.07580/) (2019)
- [Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function](https://mlsanity.com/paper/2107.08649/) (2021)
- [Chaotic Hedging with Iterated Integrals and Neural Networks](https://mlsanity.com/paper/2209.10166/) (2022)
- [Pricing options and computing implied volatilities using neural networks](https://mlsanity.com/paper/1901.08943/) (2019)
- [Deep Portfolio Optimization via Distributional Prediction of Residual Factors](https://mlsanity.com/paper/2012.07245/) (2020)

## Built on by (most cited first, 5 of 5)

- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Taming the Interacting Particle Langevin Algorithm: The Superlinear case](https://mlsanity.com/paper/2403.19587/) (2024)
- [kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradients](https://mlsanity.com/paper/2506.04878/) (2025)
- [Flatness-Aware Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2510.02174/) (2025)
- [RELTA-SGLD: Relative-Growth Localized Taming for Nonconvex Stochastic-Gradient Langevin Learning](https://mlsanity.com/paper/2607.19544/) (2026)

## Abstract (arXiv)

We introduce a new Langevin dynamics based algorithm, called e-THvarepsilonO POULA, to solve optimization problems with discontinuous stochastic gradients which naturally appear in real-world applications such as quantile estimation, vector quantization, CVaR minimization, and regularized optimization problems involving ReLU neural networks. We demonstrate both theoretically and numerically the applicability of the e-THvarepsilonO POULA algorithm. More precisely, under the conditions that the stochastic gradient is locally Lipschitz in average and satisfies a certain convexity at infinity condition, we establish non-asymptotic error bounds for e-THvarepsilonO POULA in Wasserstein distances and provide a non-asymptotic estimate for the expected excess risk, which can be controlled to be arbitrarily small. Three key applications in finance and insurance are provided, namely, multi-period portfolio optimization, transfer learning in multi-period portfolio optimization, and insurance claim prediction, which involve neural networks with (Leaky)-ReLU activation functions. Numerical experiments conducted using real-world datasets illustrate the superior empirical performance of e-THvarepsilonO POULA compared to SGLD, TUSLA, ADAM, and AMSGrad in terms of model accuracy.