mlsanityData to 14 Oct 2026

A Reinforcement Learning Framework for Some Singular Stochastic Control Problems

arXiv 2506.22203 ↗First posted 27 Jun 2025math.OCcited by 2

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We develop a continuous-time reinforcement learning framework for a class of singular stochastic control problems without entropy regularization. The optimal singular control is characterized as the optimal singular control law, which is a pair of regions of time and the augmented states. The goal of learning is to identify such an optimal region via the trial-and-error procedure. In this context, we generalize the existing policy evaluation theories with regular controls to learn our optimal singular control law and develop a policy improvement theorem via the region iteration. To facilitate the model-free policy iteration procedure, we further introduce the zero-order and first-order q-functions arising from singular control problems and establish the martingale characterization for the pair of q-functions together with the value function. Based on our theoretical findings, some q-learning algorithms are devised accordingly and a numerical example based on simulation experiment is presented.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 38 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

17 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Exploratory Optimal Stopping: A Singular Control Formulation18 Aug
  2. 2024Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty19 Apr

Other references · 15

  1. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  2. 2022q-Learning in Continuous Timecited by 91
  3. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  4. 2023Continuous-time q-learning for mean-field control problemscited by 22
  5. 2022Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithmcited by 17
  6. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 13
  7. 2024Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equationscited by 12
  8. 2022Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problemscited by 10
  9. 2023On optimal tracking portfolio in incomplete markets: The reinforcement learning approachcited by 9
  10. 2024Learning to Optimally Stop Diffusion Processes, with Financial Applicationscited by 9
  11. 2024Convergence Analysis for Entropy-Regularized Control Problems: A Probabilistic Approachcited by 6
  12. 2023Equilibria for Time-inconsistent Singular Control Problemscited by 2
3 more
  1. 2015Optimal Asset Liquidation with Multiplicative Transient Price Impactcited by 1
  2. 2018An Optimal Extraction Problem with Price Impactcited by 1
  3. 2025On the Singular Control of a Diffusion and its Running Infimum or Supremumcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Continuous-time reinforcement learning for optimal switching over multiple regimescited by 6
  2. 2025Reinforcement learning for irreversible reinsurance problems: the randomized singular control approachcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning Framework For Stochastic Optimal Control Problem Under Model Uncertainty
  2. 2026Reinforcement Learning in Real Option Models
  3. 2026Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
  4. 2026Reinforcement Learning for Discounted and Ergodic Control of Diffusion Processes
  5. 2024A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
  6. 2024Continuous-time q-Learning for Jump-Diffusion Models under Tsallis Entropy
  7. 2026Discretization error from regularized Reinforcement Learning to continuous-time stochastic control
  8. 2023Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environments