# Data-Driven Merton's Strategies via Policy Randomization

arXiv: https://arxiv.org/abs/2312.11797 · HTML: https://mlsanity.com/paper/2312.11797/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 44 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [q-Learning in Continuous Time](https://mlsanity.com/paper/2207.00713/) (2022)
- [Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach](https://mlsanity.com/paper/2108.06655/) (2021)
- [Entropy Regularization for Mean Field Games with Learning](https://mlsanity.com/paper/2010.00145/) (2020)
- [Deep Learning Approximation for Stochastic Control Problems](https://mlsanity.com/paper/1611.07422/) (2016)
- [Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications](https://mlsanity.com/paper/1812.05916/) (2018)
- [Deep Learning Statistical Arbitrage](https://mlsanity.com/paper/2106.04028/) (2021)
- [Deep Empirical Risk Minimization in finance: looking into the future](https://mlsanity.com/paper/2011.09349/) (2020)

## Built on by (most cited first, 8 of 8)

- [Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning](https://mlsanity.com/paper/2409.08400/) (2024)
- [Learning to Optimally Stop Diffusion Processes, with Financial Applications](https://mlsanity.com/paper/2408.09242/) (2024)
- [Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundations](https://mlsanity.com/paper/2604.27372/) (2026)
- [Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms](https://mlsanity.com/paper/2604.27378/) (2026)
- [Exploratory Utility Maximization Problem with Tsallis Entropy](https://mlsanity.com/paper/2502.01269/) (2025)
- [Continuous-time reinforcement learning: ellipticity enables model-free value function approximation](https://mlsanity.com/paper/2602.06930/) (2026)
- [ART for Diffusion Sampling: Continuous-Time Control and Actor-Critic Learning](https://mlsanity.com/paper/2607.02137/) (2026)
- [Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets](https://mlsanity.com/paper/2607.16625/) (2026)

## Abstract (arXiv)

We study Merton's expected utility maximization problem in an incomplete market, characterized by a factor process in addition to the stock price process, where all the model primitives are unknown. The agent under consideration is a price taker who has access only to the stock and factor value processes and the instantaneous volatility. We propose an auxiliary problem in which the agent can invoke policy randomization according to a specific class of Gaussian distributions, and prove that the mean of its optimal Gaussian policy solves the original Merton problem. With randomized policies, we are in the realm of continuous-time reinforcement learning (RL) recently developed in Wang et al. (2020) and Jia and Zhou (2022a, 2022b, 2023), enabling us to solve the auxiliary problem in a data-driven way without having to estimate the model primitives. Specifically, we establish a policy improvement theorem based on which we design both online and offline actor-critic RL algorithms for learning Merton's strategies. A key insight from this study is that RL in general and policy randomization in particular are useful beyond the purpose for exploration -- they can be employed as a technical tool to solve a problem that cannot be otherwise solved by mere deterministic policies. At last, we carry out both simulation and empirical studies in a stochastic volatility environment to demonstrate the decisive outperformance of the devised RL algorithms in comparison to the conventional model-based, plug-in method.