# A Neural RDE approach for continuous-time non-Markovian stochastic control problems

arXiv: https://arxiv.org/abs/2306.14258 · HTML: https://mlsanity.com/paper/2306.14258/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2023.

## Built on by (most cited first, 14 of 14)

- [Theoretical Foundations of Deep Selective State-Space Models](https://mlsanity.com/paper/2402.19047/) (2024)
- [Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models](https://mlsanity.com/paper/2505.17761/) (2025)
- [Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching](https://mlsanity.com/paper/2405.20799/) (2024)
- [Lecture notes on rough paths and applications to machine learning](https://mlsanity.com/paper/2404.06583/) (2024)
- [Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals](https://mlsanity.com/paper/2405.13587/) (2024)
- [Sparse Signature Coefficient Recovery via Kernels](https://mlsanity.com/paper/2412.08579/) (2024)
- [pySigLib -- Fast Signature-Based Computations on CPU and GPU](https://mlsanity.com/paper/2509.10613/) (2025)
- [Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions](https://mlsanity.com/paper/2410.03973/) (2024)
- [ParallelFlow: Parallelizing Linear Transformers via Flow Discretization](https://mlsanity.com/paper/2504.00492/) (2025)
- [Pathwise Relaxed Optimal Control of Rough Differential Equations](https://mlsanity.com/paper/2402.17900/) (2024)
- [Conservative Continuous-Time Treatment Optimization](https://mlsanity.com/paper/2603.16789/) (2026)
- [Log-PDE Methods for Rough Signature Kernels](https://mlsanity.com/paper/2404.02926/) (2024)
- [Learning non-Markovian Dynamical Systems with Signature-based Encoders](https://mlsanity.com/paper/2509.12022/) (2025)
- [Novelty detection on path space](https://mlsanity.com/paper/2512.03243/) (2025)

## Abstract (arXiv)

We propose a novel framework for solving continuous-time non-Markovian stochastic control problems by means of neural rough differential equations (Neural RDEs) introduced in Morrill et al. (2021). Non-Markovianity naturally arises in control problems due to the time delay effects in the system coefficients or the driving noises, which leads to optimal control strategies depending explicitly on the historical trajectories of the system state. By modelling the control process as the solution of a Neural RDE driven by the state process, we show that the control-state joint dynamics are governed by an uncontrolled, augmented Neural RDE, allowing for fast Monte-Carlo estimation of the value function via trajectories simulation and memory-efficient backpropagation. We provide theoretical underpinnings for the proposed algorithmic framework by demonstrating that Neural RDEs serve as universal approximators for functions of random rough paths. Exhaustive numerical experiments on non-Markovian stochastic control problems are presented, which reveal that the proposed framework is time-resolution-invariant and achieves higher accuracy and better stability in irregular sampling compared to existing RNN-based approaches.