mlsanityData to 14 Oct 2026

Risk-Sensitive Q-Learning in Continuous Time with Application to Dynamic Portfolio Selection

arXiv 2512.02386 ↗First posted 2 Dec 2025cs.LG math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper studies the problem of risk-sensitive reinforcement learning (RSRL) in continuous time, where the environment is characterized by a controllable stochastic differential equation (SDE) and the objective is a potentially nonlinear functional of cumulative rewards. We prove that when the functional is an optimized certainty equivalent (OCE), the optimal policy is Markovian with respect to an augmented environment. We also propose textitCT-RS-q, a risk-sensitive q-learning algorithm based on a novel martingale characterization approach. Finally, we run a simulation study on a dynamic portfolio selection problem and illustrate the effectiveness of our algorithm.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 22 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

14 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents10 Mar

Other references · 13

  1. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  2. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  3. 2013Risk-sensitive Reinforcement Learningcited by 130
  4. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  5. 2022q-Learning in Continuous Timecited by 91
  6. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  7. 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 72
  8. 2019Statistics and Samples in Distributional Reinforcement Learningcited by 63
  9. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 58
  10. 2021Exploratory HJB equations and their convergencecited by 32
  11. 2023Policy Optimization for Continuous Reinforcement Learningcited by 31
  12. 2020State-Dependent Temperature Control for Langevin Diffusionscited by 20
1 more
  1. 2020Minimizing Spectral Risk Measures Applied to Markov Decision Processescited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty
  2. 2026Exploratory Randomization for Discrete-Time Risk-Sensitive Benchmarked Investment Management with Reinforcement Learning
  3. 2025Robust Bayesian Dynamic Programming for On-policy Risk-sensitive Reinforcement Learning
  4. 2026Reinforcement Learning for Risk-Sensitive Investment Management: a Free Energy--Entropy Duality Approach
  5. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functions
  6. 2026Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives
  7. 2026Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets
  8. 2024Reinforcement Learning for Jump-Diffusions, with Financial Applications