# A modified MSA for stochastic control problems

arXiv: https://arxiv.org/abs/2007.05209 · HTML: https://mlsanity.com/paper/2007.05209/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee](https://mlsanity.com/paper/2302.05816/) (2023)
- [Linear convergence of a policy gradient method for some finite horizon continuous time control problems](https://mlsanity.com/paper/2203.11758/) (2022)
- [Optimal scheduling of entropy regulariser for continuous-time linear-quadratic reinforcement learning](https://mlsanity.com/paper/2208.04466/) (2022)
- [Convergence Analysis for Entropy-Regularized Control Problems: A Probabilistic Approach](https://mlsanity.com/paper/2406.10959/) (2024)
- [Adjoint Matching through the Lens of the Stochastic Maximum Principle in Optimal Control](https://mlsanity.com/paper/2604.08580/) (2026)
- [Convergence of Proximal Policy Gradient Method for Problems with Control Dependent Diffusion Coefficients](https://mlsanity.com/paper/2505.18379/) (2025)
- [Continuous Policy and Value Iteration for Relaxed Stochastic Control Problems and Its Convergence](https://mlsanity.com/paper/2506.08121/) (2025)
- [Approximation and learning with compositional tensor trains](https://mlsanity.com/paper/2512.18059/) (2025)
- [DeepPAAC: A New Deep Galerkin Method for Principal-Agent Problems](https://mlsanity.com/paper/2511.04309/) (2025)
- [Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems](https://mlsanity.com/paper/2610.04285/) (2026)

## Abstract (arXiv)

The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.