# Recurrent Neural Networks for Stochastic Control Problems with Delay

arXiv: https://arxiv.org/abs/2101.01385 · HTML: https://mlsanity.com/paper/2101.01385/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 61 references that are in the map.
- First posted on arXiv 5 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Deep neural network solution of the electronic Schr"odinger equation](https://mlsanity.com/paper/1909.08423/) (2019)
- [Deep Learning Approximation for Stochastic Control Problems](https://mlsanity.com/paper/1611.07422/) (2016)
- [Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach](https://mlsanity.com/paper/2002.02600/) (2020)
- [Deep Learning Methods for Mean Field Control Problems with Delay](https://mlsanity.com/paper/1905.00358/) (2019)
- [Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games](https://mlsanity.com/paper/1912.01809/) (2019)
- [Deep Fictitious Play for Stochastic Differential Games](https://mlsanity.com/paper/1903.09376/) (2019)
- [Convergence of Deep Fictitious Play for Stochastic Differential Games](https://mlsanity.com/paper/2008.05519/) (2020)
- [Optimal Policies for a Pandemic: A Stochastic Game Approach and a Deep Learning Algorithm](https://mlsanity.com/paper/2012.06745/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee](https://mlsanity.com/paper/2302.05816/) (2023)
- [Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms](https://mlsanity.com/paper/2307.06450/) (2023)

## Abstract (arXiv)

Stochastic control problems with delay are challenging due to the path-dependent feature of the system and thus its intrinsic high dimensions. In this paper, we propose and systematically study deep neural networks-based algorithms to solve stochastic control problems with delay features. Specifically, we employ neural networks for sequence modeling (e.g., recurrent neural networks such as long short-term memory) to parameterize the policy and optimize the objective function. The proposed algorithms are tested on three benchmark examples: a linear-quadratic problem, optimal consumption with fixed finite delay, and portfolio optimization with complete memory. Particularly, we notice that the architecture of recurrent neural networks naturally captures the path-dependent feature with much flexibility and yields better performance with more efficient and stable training of the network compared to feedforward networks. The superiority is even evident in the case of portfolio optimization with complete memory, which features infinite delay.