# Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering

arXiv: https://arxiv.org/abs/2105.08877 · HTML: https://mlsanity.com/paper/2105.08877/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Multi-step Reinforcement Learning: A Unifying Algorithm](https://mlsanity.com/paper/1703.01327/) (2017)
- [Interpretable Optimal Stopping](https://mlsanity.com/paper/1812.07211/) (2018)
- [Deep Learning for Ranking Response Surfaces with Applications to Optimal Stopping Problems](https://mlsanity.com/paper/1901.03478/) (2019)
- [Sample Efficient Policy Search for Optimal Stopping Domains](https://mlsanity.com/paper/1702.06238/) (2017)
- [Zap Q-Learning for Optimal Stopping Time Problems](https://mlsanity.com/paper/1904.11538/) (2019)

## Built on by (most cited first, 6 of 6)

- [Timing is Everything: Learning to Act Selectively with Costly Actions and Budgetary Constraints](https://mlsanity.com/paper/2205.15953/) (2022)
- [Deep Learning for the Multiple Optimal Stopping Problem](https://mlsanity.com/paper/2512.22961/) (2025)
- [Learning to Liquidate Forex: Optimal Stopping via Adaptive Top-K Regression](https://mlsanity.com/paper/2202.12578/) (2022)
- [Risk-averse policies for natural gas futures trading using distributional reinforcement learning](https://mlsanity.com/paper/2501.04421/) (2025)
- [Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review](https://mlsanity.com/paper/2510.21758/) (2025)
- [Continuous-time Optimal Stopping through Deep Reinforcement Learning](https://mlsanity.com/paper/2606.17545/) (2026)

## Abstract (arXiv)

Optimal stopping is the problem of deciding the right time at which to take a particular action in a stochastic system, in order to maximize an expected reward. It has many applications in areas such as finance, healthcare, and statistics. In this paper, we employ deep Reinforcement Learning (RL) to learn optimal stopping policies in two financial engineering applications: namely option pricing, and optimal option exercise. We present for the first time a comprehensive empirical evaluation of the quality of optimal stopping policies identified by three state of the art deep RL algorithms: double deep Q-learning (DDQN), categorical distributional RL (C51), and Implicit Quantile Networks (IQN). In the case of option pricing, our findings indicate that in a theoretical Black-Schole environment, IQN successfully identifies nearly optimal prices. On the other hand, it is slightly outperformed by C51 when confronted to real stock data movements in a put option exercise problem that involves assets from the S&P500 index. More importantly, the C51 algorithm is able to identify an optimal stopping policy that achieves 8% more out-of-sample returns than the best of four natural benchmark policies. We conclude with a discussion of our findings which should pave the way for relevant future research.