# A Forward Propagation Algorithm for Online Optimization of Nonlinear Stochastic Differential Equations

arXiv: https://arxiv.org/abs/2207.04496 · HTML: https://mlsanity.com/paper/2207.04496/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 10 Jul 2022.

## Builds on (most cited first, 9 of 9)

- [Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II -- The Finite Horizon Case](https://mlsanity.com/paper/1908.01613/) (2019)
- [Stochastic Gradient Descent in Continuous Time](https://mlsanity.com/paper/1611.05545/) (2016)
- [Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem](https://mlsanity.com/paper/1710.04273/) (2017)
- [A Continuous-time Stochastic Gradient Descent Method for Continuous Data](https://mlsanity.com/paper/2112.03754/) (2021)
- [Online Maximum Likelihood Estimation of the Parameters of Partially Observed Diffusion Processes](https://mlsanity.com/paper/1611.00170/) (2016)
- [Two-Timescale Stochastic Gradient Descent in Continuous Time with Applications to Joint Online Parameter Estimation and Optimal Sensor Placement](https://mlsanity.com/paper/2007.15998/) (2020)
- [Continuous-time stochastic gradient descent for optimizing over the stationary distribution of stochastic differential equations](https://mlsanity.com/paper/2202.06637/) (2022)
- [Global Convergence of the ODE Limit for Online Actor-Critic Algorithms in Reinforcement Learning](https://mlsanity.com/paper/2108.08655/) (2021)
- [Online Adjoint Methods for Optimization of PDEs](https://mlsanity.com/paper/2101.09621/) (2021)

## Built on by (most cited first, 4 of 4)

- [Implicit Diffusion: Efficient Optimization through Stochastic Sampling](https://mlsanity.com/paper/2402.05468/) (2024)
- [An Efficient High-Dimensional Gradient Estimator for Stochastic Differential Equations](https://mlsanity.com/paper/2407.10065/) (2024)
- [OGF: An Online Gradient Flow Method for Optimizing the Statistical Steady-State Time Averages of Unsteady Turbulent Flows](https://mlsanity.com/paper/2507.05149/) (2025)
- [Recursive Maximum Likelihood Estimation for Interacting Particle Systems using Virtual Particles](https://mlsanity.com/paper/2605.00786/) (2026)

## Abstract (arXiv)

Optimizing over the stationary distribution of stochastic differential equations (SDEs) is computationally challenging. A new forward propagation algorithm has been recently proposed for the online optimization of SDEs. The algorithm solves an SDE, derived using forward differentiation, which provides a stochastic estimate for the gradient. The algorithm continuously updates the SDE model's parameters and the gradient estimate simultaneously. This paper studies the convergence of the forward propagation algorithm for nonlinear dissipative SDEs. We leverage the ergodicity of this class of nonlinear SDEs to characterize the convergence rate of the transition semi-group and its derivatives. Then, we prove bounds on the solution of a Poisson partial differential equation (PDE) for the expected time integral of the algorithm's stochastic fluctuations around the direction of steepest descent. We then re-write the algorithm using the PDE solution, which allows us to characterize the parameter evolution around the direction of steepest descent. Our main result is a convergence theorem for the forward propagation algorithm for nonlinear dissipative SDEs.