# Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions

arXiv: https://arxiv.org/abs/2608.19151 · HTML: https://mlsanity.com/paper/2608.19151/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 54 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 11 of 11)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [SIR-Hawkes: Linking Epidemic Models and Hawkes Processes to Model Diffusions in Finite Populations](https://mlsanity.com/paper/1711.01679/) (2017)
- [Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State](https://mlsanity.com/paper/2509.23711/) (2025)
- [Flexible Spatio-Temporal Hawkes Process Models for Earthquake Occurrences](https://mlsanity.com/paper/2210.08053/) (2022)
- [Gradient-based estimation of linear Hawkes processes with general kernels](https://mlsanity.com/paper/2111.10637/) (2021)
- [Optimal market making with persistent order flow](https://mlsanity.com/paper/2003.05958/) (2020)
- [Markov approximation for controlled Hawkes Jump-Diffusions with general kernels](https://mlsanity.com/paper/2507.11294/) (2025)
- [Learning Market Making with Closing Auctions](https://mlsanity.com/paper/2601.17247/) (2026)
- [Approximation of Singular-Stopping Control Driven by Hawkes Processes via Rescaled MDPs](https://mlsanity.com/paper/2602.05025/) (2026)
- [Signature Methods for Optimal Market Making](https://mlsanity.com/paper/2606.19772/) (2026)

## Built on by (most cited first, 1 of 1)

- [When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk](https://mlsanity.com/paper/2609.32083/) (2026)

## Abstract (arXiv)

We study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels. We first develop a finite-dimensional Markovianization procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels. We prove the convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the value of the primal problem. We then formulate continuous-time deterministic policy gradient learning on the Markovianized approximation of the problem, called Hawkes-CT DDPG. We propose a model-free algorithm to solve the non-Markovian Hawkes-driven optimization by observing only the event times of the process, the realization of the solution to the SDE, and a chosen set of decay filters, while the Hawkes kernel coefficients remain unknown. We compare our continuous time reinforcement learning Hawkes-CT DDPG method with discrete time reinforcement learning techniques under three different types of kernels: simple exponential, Erlang, and power-law kernels.