mlsanityData to 14 Oct 2026

Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing

arXiv 2608.20680 ↗First posted 21 Aug 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We study reinforcement learning (RL) in Continuous-Time Jump Markov Decision Processes (CTJMDPs) featuring general discrete state spaces (which need not possess a vector space structure) and continuous/discrete action spaces. The setup covers many well-known applications in operations such as multi-product dynamic pricing with capacitated resources (Gallego and van Ryzin 1997). To model the exploration-exploitation tradeoff, we formulate an entropy-regularized continuous-time control problem with stochastic policies. Recent continuous-time RL techniques such as q-learning for controlled diffusions in (Jia and Zhou 2023) focus on continuous state spaces ℝd and rely heavily on semimartingale theory in ℝd for their theoretical analysis. Consequently, their methods cannot be directly applied to CTJMDPs with general discrete state spaces, which may lack the algebraic addition and subtraction structures inherent to Euclidean spaces. To bridge this gap, we establish the theoretical foundations of q-learning for CTJMDPs and develop model-free q-learning algorithms. Compared to na"ive time discretization and approximating CTJMDPs using discrete-time MDPs, our approach has several conceptual and empirical benefits. Numerical experiments in network dynamic pricing (Gallego and van Ryzin 1997) show that our proposed RL algorithm reliably learns near-optimal policies and consistently outperforms standard benchmark methods, demonstrating superior solution quality and effective scalability to large-scale network instances.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 42 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

12 of 42 references.

References · 12

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2022q-Learning in Continuous Timecited by 91
  3. 2019Making Deep Q-learning methods robust to time discretizationcited by 58
  4. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
  5. 2020Queueing Network Controls via Deep Reinforcement Learningcited by 41
  6. 2023Policy Optimization for Continuous Reinforcement Learningcited by 31
  7. 2021Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methodscited by 23
  8. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 13
  9. 2022Logarithmic regret bounds for continuous-time average-reward Markov decision processescited by 8
  10. 2022Square-root regret bounds for continuous-time episodic Markov decision processescited by 5
  11. 2024Reinforcement Learning for Intensity Control: An Application to Choice-Based Network Revenue Managementcited by 3
  12. 2026A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Reinforcement Learning for Jump-Diffusions, with Financial Applications
  2. 2024A Course in Dynamic Optimization
  3. 2025Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control
  4. 2025Continuous-time reinforcement learning for optimal switching over multiple regimes
  5. 2021Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuity
  6. 2024Continuous-time q-Learning for Jump-Diffusion Models under Tsallis Entropy
  7. 2026Q-Measure-Learning for Continuous State RL: Efficient Implementation and Convergence
  8. 2026An Actor-Critic Framework for Continuous-Time Jump-Diffusion Controls with Normalizing Flows