# Neural Controlled Differential Equations for Online Prediction Tasks

arXiv: https://arxiv.org/abs/2106.11028 · HTML: https://mlsanity.com/paper/2106.11028/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks](https://mlsanity.com/paper/1902.09689/) (2019)
- [Neural Spatio-Temporal Point Processes](https://mlsanity.com/paper/2011.04583/) (2020)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Deep Signature Transforms](https://mlsanity.com/paper/1905.08494/) (2019)
- [On Second Order Behaviour in Augmented Neural ODEs](https://mlsanity.com/paper/2006.07220/) (2020)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU](https://mlsanity.com/paper/2001.00706/) (2020)
- [MALI: A memory efficient and reverse accurate integrator for Neural ODEs](https://mlsanity.com/paper/2102.04668/) (2021)
- ["Hey, that's not an ODE": Faster ODE Adjoints via Seminorms](https://mlsanity.com/paper/2009.09457/) (2020)
- [Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering](https://mlsanity.com/paper/2006.04727/) (2020)
- [Policy Analysis using Synthetic Controls in Continuous-Time](https://mlsanity.com/paper/2102.01577/) (2021)
- [Monash University, UEA, UCR Time Series Extrinsic Regression Archive](https://mlsanity.com/paper/2006.10996/) (2020)
- [A Generalised Signature Method for Multivariate Time Series Feature Extraction](https://mlsanity.com/paper/2006.00873/) (2020)

## Built on by (most cited first, 30 of 46)

- [Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations](https://mlsanity.com/paper/2206.08311/) (2022)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [Bayesian Neural Controlled Differential Equations for Treatment Effect Estimation](https://mlsanity.com/paper/2310.17463/) (2023)
- [Neural signature kernels as infinite-width-depth-limits of controlled ResNets](https://mlsanity.com/paper/2303.17671/) (2023)
- [EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecasting](https://mlsanity.com/paper/2204.08771/) (2022)
- [Comprehensive Review of Neural Differential Equations for Time Series Analysis](https://mlsanity.com/paper/2502.09885/) (2025)
- [Estimating Treatment Effects from Irregular Time Series Observations with Hidden Confounders](https://mlsanity.com/paper/2303.02320/) (2023)
- [Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules](https://mlsanity.com/paper/2206.01649/) (2022)
- [Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Time](https://mlsanity.com/paper/2306.04255/) (2023)
- [New directions in the applications of rough path theory](https://mlsanity.com/paper/2302.04586/) (2023)
- [Graph-based Multi-ODE Neural Networks for Spatio-Temporal Traffic Forecasting](https://mlsanity.com/paper/2305.18687/) (2023)
- [Stop&Hop: Early Classification of Irregular Time Series](https://mlsanity.com/paper/2208.09795/) (2022)
- [Efficient, Accurate and Stable Gradients for Neural ODEs](https://mlsanity.com/paper/2410.11648/) (2024)
- [Learnable Path in Neural Controlled Differential Equations](https://mlsanity.com/paper/2301.04333/) (2023)
- [Tracing Agentic Failure from the Flow of Success](https://mlsanity.com/paper/2607.12747/) (2026)
- [Dynamical Survival Analysis with Controlled Latent States](https://mlsanity.com/paper/2401.17077/) (2024)
- [Addressing Prediction Delays in Time Series Forecasting: A Continuous GRU Approach with Derivative Regularization](https://mlsanity.com/paper/2407.01622/) (2024)
- [Stabilized Neural Prediction of Potential Outcomes in Continuous Time](https://mlsanity.com/paper/2410.03514/) (2024)
- [Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learning](https://mlsanity.com/paper/2301.05664/) (2023)
- [TS-Diffusion: Generating Highly Complex Time Series with Diffusion Models](https://mlsanity.com/paper/2311.03303/) (2023)
- [Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines](https://mlsanity.com/paper/2410.13563/) (2024)
- [HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs](https://mlsanity.com/paper/2506.00088/) (2025)
- [DeNOTS: Stable Deep Neural ODEs for Time Series](https://mlsanity.com/paper/2408.08055/) (2024)
- [Multiscale Sensor Fusion and Continuous Control with Neural CDEs](https://mlsanity.com/paper/2203.08715/) (2022)
- [Hybridizing Physics and Neural ODEs for Predicting Plasma Inductance Dynamics in Tokamak Fusion Reactors](https://mlsanity.com/paper/2310.20079/) (2023)
- [TrajSurv: Learning Continuous Latent Trajectories from Electronic Health Records for Trustworthy Survival Prediction](https://mlsanity.com/paper/2508.00657/) (2025)
- [Continuous-Time Piecewise-Linear Recurrent Neural Networks](https://mlsanity.com/paper/2602.15649/) (2026)
- [On the effectiveness of Randomized Signatures as Reservoir for Learning Rough Dynamics](https://mlsanity.com/paper/2201.00384/) (2022)
- [Learning the Dynamics of Sparsely Observed Interacting Systems](https://mlsanity.com/paper/2301.11647/) (2023)

## Abstract (arXiv)

Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in textitonline prediction tasks, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks.