# Reinforcement Learning for Risk-Sensitive Investment Management: a Free Energy--Entropy Duality Approach

arXiv: https://arxiv.org/abs/2606.20903 · HTML: https://mlsanity.com/paper/2606.20903/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 8 of 8)

- [Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms](https://mlsanity.com/paper/2111.11232/) (2021)
- [q-Learning in Continuous Time](https://mlsanity.com/paper/2207.00713/) (2022)
- [Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach](https://mlsanity.com/paper/2108.06655/) (2021)
- [Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty](https://mlsanity.com/paper/2404.12598/) (2024)
- [Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State](https://mlsanity.com/paper/2509.23711/) (2025)
- [On the Separation of Estimation and Control in Risk-Sensitive Investment Problems under Incomplete Observation](https://mlsanity.com/paper/2304.08910/) (2023)
- [Exploratory Randomization for Discrete-Time Risk-Sensitive Benchmarked Investment Management with Reinforcement Learning](https://mlsanity.com/paper/2603.00738/) (2026)
- [Risk-Sensitive Investment Management via Free Energy-Entropy Duality](https://mlsanity.com/paper/2604.15463/) (2026)

## Abstract (arXiv)

This paper develops a reinforcement-learning approach to continuous-time risk-sensitive benchmarked asset allocation in a partly model-based setting. The benchmarked problem does not directly fit the standard Markovian stochastic-control template: the state is uncontrolled, whereas the terminal reward contains a controlled Ito integral. We use free energy-entropy duality to reformulate the problem as a linear-quadratic-Gaussian stochastic differential game under an equivalent probability measure, yielding explicit finite- and infinite-horizon saddle-point solutions. This structure guides a continuous-time q-learning actor-critic method: the quadratic value function motivates the critic, while the affine saddle-point controls motivate deterministic actors for the portfolio allocation and adversarial control. The learned allocation admits an economic interpretation through fractional Kelly decompositions. A proof-of-concept implementation calibrated to U.S. equity data shows that the actors learn the optimal policy with high accuracy and reveals a favorable asymmetry: the portfolio actor receives a cleaner learning signal than the auxiliary adversarial actor.