# Continuous-time stochastic gradient descent for optimizing over the stationary distribution of stochastic differential equations

arXiv: https://arxiv.org/abs/2202.06637 · HTML: https://mlsanity.com/paper/2202.06637/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 81 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Neural Spatio-Temporal Point Processes](https://mlsanity.com/paper/2011.04583/) (2020)
- [Sig-Wasserstein GANs for Time Series Generation](https://mlsanity.com/paper/2111.01207/) (2021)
- [Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon](https://mlsanity.com/paper/2011.10300/) (2020)
- [Robust pricing and hedging via neural SDEs](https://mlsanity.com/paper/2007.04154/) (2020)
- [Deep Learning for Mean Field Games and Mean Field Control with Applications to Finance](https://mlsanity.com/paper/2107.04568/) (2021)
- [Arbitrage-free neural-SDE market models](https://mlsanity.com/paper/2105.11053/) (2021)
- [Stochastic Gradient Descent in Continuous Time](https://mlsanity.com/paper/1611.05545/) (2016)
- [Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem](https://mlsanity.com/paper/1710.04273/) (2017)
- [Parameter Estimation for the McKean-Vlasov Stochastic Differential Equation](https://mlsanity.com/paper/2106.13751/) (2021)
- [Online Maximum Likelihood Estimation of the Parameters of Partially Observed Diffusion Processes](https://mlsanity.com/paper/1611.00170/) (2016)
- [Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: I -- The Ergodic Case](https://mlsanity.com/paper/1907.05980/) (2019)
- [Two-Timescale Stochastic Gradient Descent in Continuous Time with Applications to Joint Online Parameter Estimation and Optimal Sensor Placement](https://mlsanity.com/paper/2007.15998/) (2020)
- [Linear-Quadratic N-person and Mean-Field Games with Ergodic Cost](https://mlsanity.com/paper/1401.1421/) (2014)
- [Joint Online Parameter Estimation and Optimal Sensor Placement for the Partially Observed Stochastic Advection-Diffusion Equation](https://mlsanity.com/paper/2009.08693/) (2020)
- [Deep Hawkes Process for High-Frequency Market Making](https://mlsanity.com/paper/2109.15110/) (2021)
- [Mean Reversion Trading with Sequential Deadlines and Transaction Costs](https://mlsanity.com/paper/1707.03498/) (2017)
- [Mean Reverting Portfolios via Penalized OU-Likelihood Estimation](https://mlsanity.com/paper/1803.06460/) (2018)
- [Ergodic behavior of control and mean field games problems depending on acceleration](https://mlsanity.com/paper/2007.02704/) (2020)
- [Stationary Discounted and Ergodic Mean Field Games of Singular Control](https://mlsanity.com/paper/2105.07213/) (2021)

## Built on by (most cited first, 4 of 4)

- [Implicit Diffusion: Efficient Optimization through Stochastic Sampling](https://mlsanity.com/paper/2402.05468/) (2024)
- [A Forward Propagation Algorithm for Online Optimization of Nonlinear Stochastic Differential Equations](https://mlsanity.com/paper/2207.04496/) (2022)
- [OGF: An Online Gradient Flow Method for Optimizing the Statistical Steady-State Time Averages of Unsteady Turbulent Flows](https://mlsanity.com/paper/2507.05149/) (2025)
- [Recursive Maximum Likelihood Estimation for Interacting Particle Systems using Virtual Particles](https://mlsanity.com/paper/2605.00786/) (2026)

## Abstract (arXiv)

We develop a new continuous-time stochastic gradient descent method for optimizing over the stationary distribution of stochastic differential equation (SDE) models. The algorithm continuously updates the SDE model's parameters using an estimate for the gradient of the stationary distribution. The gradient estimate is simultaneously updated using forward propagation of the SDE state derivatives, asymptotically converging to the direction of steepest descent. We rigorously prove convergence of the online forward propagation algorithm for linear SDE models (i.e., the multi-dimensional Ornstein-Uhlenbeck process) and present its numerical results for nonlinear examples. The proof requires analysis of the fluctuations of the parameter evolution around the direction of steepest descent. Bounds on the fluctuations are challenging to obtain due to the online nature of the algorithm (e.g., the stationary distribution will continuously change as the parameters change). We prove bounds for the solutions of a new class of Poisson partial differential equations (PDEs), which are then used to analyze the parameter fluctuations in the algorithm. Our algorithm is applicable to a range of mathematical finance applications involving statistical calibration of SDE models and stochastic optimal control for long time horizons where ergodicity of the data and stochastic process is a suitable modeling framework. Numerical examples explore these potential applications, including learning a neural network control for high-dimensional optimal control of SDEs and training stochastic point process models of limit order book events.