# Learning to Optimally Stop Diffusion Processes, with Financial Applications

arXiv: https://arxiv.org/abs/2408.09242 · HTML: https://mlsanity.com/paper/2408.09242/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 17 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 11 of 11)

- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms](https://mlsanity.com/paper/2111.11232/) (2021)
- [q-Learning in Continuous Time](https://mlsanity.com/paper/2207.00713/) (2022)
- [Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach](https://mlsanity.com/paper/2108.06655/) (2021)
- [Making Deep Q-learning methods robust to time discretization](https://mlsanity.com/paper/1901.09732/) (2019)
- [Deep learning of free boundary and Stefan problems](https://mlsanity.com/paper/2006.05311/) (2020)
- [Optimal Stopping via Randomized Neural Networks](https://mlsanity.com/paper/2104.13669/) (2021)
- [Solving high-dimensional optimal stopping problems using deep learning](https://mlsanity.com/paper/1908.01602/) (2019)
- [State-Dependent Temperature Control for Langevin Diffusions](https://mlsanity.com/paper/2011.07456/) (2020)
- [Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.02409/) (2022)
- [Data-Driven Merton's Strategies via Policy Randomization](https://mlsanity.com/paper/2312.11797/) (2023)

## Built on by (most cited first, 9 of 9)

- [Continuous-time reinforcement learning for optimal switching over multiple regimes](https://mlsanity.com/paper/2512.04697/) (2025)
- [Deterministic Policy Gradient for Learning Equilibrium in Time-Inconsistent Control Problems](https://mlsanity.com/paper/2606.11798/) (2026)
- [A Reinforcement Learning Framework for Some Singular Stochastic Control Problems](https://mlsanity.com/paper/2506.22203/) (2025)
- [Reinforcement learning for irreversible reinsurance problems: the randomized singular control approach](https://mlsanity.com/paper/2512.02769/) (2025)
- [Robust Exploratory Stopping under Ambiguity in Reinforcement Learning](https://mlsanity.com/paper/2510.10260/) (2025)
- [Reinforcement Learning for Speculative Trading under Exploratory Framework](https://mlsanity.com/paper/2604.02035/) (2026)
- [Randomized Optimal Switching Problem and Related Mirror Descent Flow](https://mlsanity.com/paper/2606.12875/) (2026)
- [Continuous-time Optimal Stopping through Deep Reinforcement Learning](https://mlsanity.com/paper/2606.17545/) (2026)
- [Deep Learning for Reflected BSDEs: Regularization and Error Analysis](https://mlsanity.com/paper/2609.05434/) (2026)

## Abstract (arXiv)

We study optimal stopping for diffusion processes with unknown model primitives within the continuous-time reinforcement learning (RL) framework developed by Wang et al. (2020), and present applications to option pricing and portfolio choice. By penalizing the corresponding variational inequality formulation, we transform the stopping problem into a stochastic optimal control problem with two actions. We then randomize controls into Bernoulli distributions and add an entropy regularizer to encourage exploration. We derive a semi-analytical optimal Bernoulli distribution, based on which we devise RL algorithms using the martingale approach established in Jia and Zhou (2022a). We establish a policy improvement theorem and prove the fast convergence of the resulting policy iterations. We demonstrate the effectiveness of the algorithms in pricing finite-horizon American put options, solving Merton's problem with transaction costs, and scaling to high-dimensional optimal stopping problems. In particular, we show that both the offline and online algorithms achieve high accuracy in learning the value functions and characterizing the associated free boundaries.