mlsanityData to 14 Oct 2026

Randomized Optimal Switching Problem and Related Mirror Descent Flow

arXiv 2606.12875 ↗First posted 11 Jun 2026math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We study continuous-time reinforcement learning for the optimal switching problem, in which a decision-maker controls a diffusion process by switching among finitely many regimes, incurring both running and transition costs. To enable exploration, we relax the classical deterministic switching control to a randomized framework, where the switching decisions are governed by a continuous-time Markov chain with state-dependent generator, and augment the cost functional with a KL-divergence regularization weighted by a temperature parameter λ. Under mild assumptions on the coefficients, we establish that the regularized value function is the unique smooth solution of an elliptic Hamilton--Jacobi--Bellman system, and derive an explicit optimal Gibbs policy given by an exponential transformation of the value function differences across modes. We further prove that the regularized value function approximates the classical optimal value function with error of order Oleft(λ log frac1λright), which is consistent with analogous bounds established in other entropy-regularized control problems and is believed to be sharp. To solve the regularized problem numerically, we introduce a mirror descent flow in the dual logarithmic policy space, prove its well-posedness and the monotonic decrease of the value function along the flow, and establish quantitative error bound to the classical optimal value function. For a constant temperature scheduler, the convergence rate is of order Oleft(frac1eλ s - 1+λ logfrac1λright), while under the annealing scheduler λs = frac1√1+s, we obtain the rate Oleft(fraclog s√sright), which decays to zero as the flow time s → ∞.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 30 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

14 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Continuous-time reinforcement learning for optimal switching over multiple regimes4 Dec
  2. 2024Exploratory Optimal Stopping: A Singular Control Formulation18 Aug

Other references · 12

  1. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 177
  2. 2021Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classescited by 156
  3. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  4. 2022q-Learning in Continuous Timecited by 91
  5. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  6. 2021Exploratory HJB equations and their convergencecited by 32
  7. 2023A Fisher-Rao gradient flow for entropy-regularised Markov decision processes in Polish spacescited by 23
  8. 2022Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithmcited by 17
  9. 2022Convergence Rates for Regularized Optimal Transport via Quantizationcited by 12
  10. 2024Entropy annealing for policy mirror descent in continuous time and spacecited by 12
  11. 2024Learning to Optimally Stop Diffusion Processes, with Financial Applicationscited by 9
  12. 2022A neural network approach to high-dimensional optimal switching problems with jumps in energy marketscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning in Real Option Models
  2. 2024Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching
  3. 2024Mirror Descent for Stochastic Control Problems with Measure-valued Controls
  4. 2026Reinforcement Learning for Discounted and Ergodic Control of Diffusion Processes
  5. 2026Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process
  6. 2024Optimal Rates of Convergence for Entropy Regularization in Discounted Markov Decision Processes
  7. 2025Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost
  8. 2026On the Policy Convergence of Policy Mirror Descent Methods