# Neural delay differential equations: learning non-Markovian closures for partially known dynamical systems

arXiv: https://arxiv.org/abs/2410.02843 · HTML: https://mlsanity.com/paper/2410.02843/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 60 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Latent ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/1907.03907/) (2019)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Neural Delay Differential Equations](https://mlsanity.com/paper/2102.10801/) (2021)
- [Neural Closure Models for Dynamical Systems](https://mlsanity.com/paper/2012.13869/) (2020)
- [Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems](https://mlsanity.com/paper/2110.14296/) (2021)
- [Neural Integro-Differential Equations](https://mlsanity.com/paper/2206.14282/) (2022)
- [Mori-Zwanzig latent space Koopman closure for nonlinear autoencoder](https://mlsanity.com/paper/2310.10745/) (2023)
- [CD-ROM: Complemented Deep-Reduced Order Model](https://mlsanity.com/paper/2202.10746/) (2022)
- [Time and State Dependent Neural Delay Differential Equations](https://mlsanity.com/paper/2306.14545/) (2023)
- [Learning the Delay Using Neural Delay Differential Equations](https://mlsanity.com/paper/2304.01329/) (2023)

## Built on by (most cited first, 1 of 1)

- [Learning non-Markovian Dynamical Systems with Signature-based Encoders](https://mlsanity.com/paper/2509.12022/) (2025)

## Abstract (arXiv)

Recent advances in learning dynamical systems from data have shown significant promise. However, many existing methods assume access to the full state of the system -- an assumption that is rarely satisfied in practice, where systems are typically monitored through a limited number of sensors, leading to partial observability. To address this challenge, we draw inspiration from the Mori-Zwanzig formalism, which provides a theoretical connection between hidden variables and memory terms. Motivated by this perspective, we introduce a constant-lag Neural Delay Differential Equations (NDDEs) framework, providing a continuous-time approach for learning non-Markovian dynamics directly from data. These memory effects are captured using a finite set of time delays, which are identified via the adjoint method. We validate the proposed approach on a range of datasets, including synthetic systems, chaotic dynamics, and experimental measurements, such as the Kuramoto-Sivashinsky equation and cavity-flow experiments. Results demonstrate that NDDEs compare favourably with existing approaches for partially observed systems, including long short-term memory (LSTM) networks and augmented neural ordinary differential equations (ANODEs). Overall, NDDEs offer a principled and data-efficient framework for modelling non-Markovian dynamics under partial observability. An open-source implementation accompanies this article.