# Reinforcement Learning in Real Option Models

arXiv: https://arxiv.org/abs/2602.15643 · HTML: https://mlsanity.com/paper/2602.15643/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 19 of 19)

- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Optimal rates for zero-order convex optimization: the power of two function evaluations](https://mlsanity.com/paper/1312.2139/) (2013)
- [Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon](https://mlsanity.com/paper/2011.10300/) (2020)
- [Dealing with Sparse Rewards in Reinforcement Learning](https://mlsanity.com/paper/1910.09281/) (2019)
- [Optimal Stopping via Randomized Neural Networks](https://mlsanity.com/paper/2104.13669/) (2021)
- [Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.02409/) (2022)
- [Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations](https://mlsanity.com/paper/2406.00612/) (2024)
- [Exploratory Optimal Stopping: A Singular Control Formulation](https://mlsanity.com/paper/2408.09335/) (2024)
- [Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching](https://mlsanity.com/paper/2404.17939/) (2024)
- [Randomized optimal stopping algorithms and their convergence analysis](https://mlsanity.com/paper/2002.00816/) (2020)
- [Learning to reflect: A unifying approach for data-driven stochastic control strategies](https://mlsanity.com/paper/2104.11496/) (2021)
- [Data-driven rules for multidimensional reflection problems](https://mlsanity.com/paper/2311.06639/) (2023)
- [Reinforcement Learning for optimal dividend problem under diffusion model](https://mlsanity.com/paper/2309.10242/) (2023)
- [Data-driven optimal stopping: A pure exploration analysis](https://mlsanity.com/paper/2312.05880/) (2023)
- [Singular Control of (Reflected) Brownian Motion: A Computational Method Suitable for Queueing Applications](https://mlsanity.com/paper/2312.11823/) (2023)
- [A solvable two-dimensional singular stochastic control problem with non convex costs](https://mlsanity.com/paper/1411.2428/) (2014)
- [Irreversible Investment under L'evy Uncertainty: an Equation for the Optimal Boundary](https://mlsanity.com/paper/1411.2395/) (2014)
- [Decision Making under Costly Sequential Information Acquisition: the Paradigm of Reversible and Irreversible Decisions](https://mlsanity.com/paper/2401.00569/) (2023)

## Abstract (arXiv)

We investigate an entropy-regularized reinforcement learning (RL) approach to optimal stopping problems motivated by real option models. Classical stopping rules are strict and non-randomized, limiting natural exploration in RL settings. To address this, we introduce entropy regularization, allowing randomized stopping policies that balance exploitation and exploration. We derive an explicit analytical solution to the regularized problem and prove convergence of the associated free boundary to the classical stopping threshold as the entropy vanishes. The regularized problem admits a natural formulation as a singular stochastic control problem. Building on this structure, we propose both model-based and model-free policy iteration algorithms to learn the optimal boundary. The model-free method operates without knowledge of system dynamics, using only trajectories from the stochastic environment. We establish convergence guarantees and illustrate strong numerical performance. This framework provides a principled and tractable approach for data-driven stopping problems under uncertainty.