# Neural Rough Differential Equations for Long Time Series

arXiv: https://arxiv.org/abs/2009.08295 · HTML: https://mlsanity.com/paper/2009.08295/

## Key facts

- Cited by 143 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 30 of 143)

- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models](https://mlsanity.com/paper/2208.09399/) (2022)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [Modeling Irregular Time Series with Continuous Recurrent Units](https://mlsanity.com/paper/2111.11344/) (2021)
- [Theoretical Foundations of Deep Selective State-Space Models](https://mlsanity.com/paper/2402.19047/) (2024)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Effectively Modeling Time Series with Simple Discrete State Spaces](https://mlsanity.com/paper/2303.09489/) (2023)
- [Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics](https://mlsanity.com/paper/2110.10249/) (2021)
- [Neural Controlled Differential Equations for Online Prediction Tasks](https://mlsanity.com/paper/2106.11028/) (2021)
- [Parallelizing non-linear sequential models over the sequence length](https://mlsanity.com/paper/2309.12252/) (2023)
- [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021)
- [Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning](https://mlsanity.com/paper/2106.09004/) (2021)
- [Oscillatory State-Space Models](https://mlsanity.com/paper/2410.03943/) (2024)
- [Non-adversarial training of Neural SDEs with signature kernel scores](https://mlsanity.com/paper/2305.16274/) (2023)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference](https://mlsanity.com/paper/2402.18512/) (2024)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [Framing RNN as a kernel method: A neural ODE approach](https://mlsanity.com/paper/2106.01202/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Comprehensive Review of Neural Differential Equations for Time Series Analysis](https://mlsanity.com/paper/2502.09885/) (2025)
- [SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputation](https://mlsanity.com/paper/2410.13338/) (2024)
- [Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models](https://mlsanity.com/paper/2505.17761/) (2025)
- [S7: Selective and Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2410.03464/) (2024)
- [Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules](https://mlsanity.com/paper/2206.01649/) (2022)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching](https://mlsanity.com/paper/2405.20799/) (2024)

## Abstract (arXiv)

Neural controlled differential equations (CDEs) are the continuous-time analogue of recurrent neural networks, as Neural ODEs are to residual networks, and offer a memory-efficient continuous-time way to model functions of potentially irregular time series. Existing methods for computing the forward pass of a Neural CDE involve embedding the incoming time series into path space, often via interpolation, and using evaluations of this path to drive the hidden state. Here, we use rough path theory to extend this formulation. Instead of directly embedding into path space, we instead represent the input signal over small time intervals through its textitlog-signature, which are statistics describing how the signal drives a CDE. This is the approach for solving textitrough differential equations (RDEs), and correspondingly we describe our main contribution as the introduction of Neural RDEs. This extension has a purpose: by generalising the Neural CDE approach to a broader class of driving signals, we demonstrate particular advantages for tackling long time series. In this regime, we demonstrate efficacy on problems of length up to 17k observations and observe significant training speed-ups, improvements in model performance, and reduced memory requirements compared to existing approaches.