mlsanityData to 14 Oct 2026

On the Generalization and Approximation Capacities of Neural Controlled Differential Equations

arXiv 2305.16791 ↗First posted 26 May 2023stat.ML cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Neural Controlled Differential Equations (NCDEs) are a state-of-the-art tool for supervised learning with irregularly sampled time series (Kidger, 2020). However, no theoretical analysis of their performance has been provided yet, and it remains unclear in particular how the irregularity of the time series affects their predictions. By merging the rich theory of controlled differential equations (CDE) and Lipschitz-based measures of the complexity of deep neural nets, we take a first step towards the theoretical understanding of NCDE. Our first result is a generalization bound for this class of predictors that depends on the regularity of the time series data. In a second time, we leverage the continuity of the flow of CDEs to provide a detailed analysis of both the sampling-induced bias and the approximation bias. Regarding this last result, we show how classical approximation results on neural nets may transfer to NCDEs. Our theoretical results are validated through a series of experiments.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 58 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

35 of 58 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  5. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  6. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  7. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  8. 2019Augmented Neural ODEscited by 397
  9. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  10. 2022On Neural Differential Equationscited by 268
  11. 2019Residual Flows for Invertible Generative Modelingcited by 257
  12. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
23 more
  1. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  2. 2020Dissecting Neural ODEscited by 153
  3. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
  4. 2016A Primer on the Signature Method in Machine Learningcited by 104
  5. 2019Orthogonal Deep Neural Networkscited by 72
  6. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  7. 2021The Modern Mathematics of Deep Learningcited by 63
  8. 2022Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equationscited by 59
  9. 2021Neural ODE control for classification, approximation and transportcited by 57
  10. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  11. 2021Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progressioncited by 48
  12. 2021Neural Controlled Differential Equations for Online Prediction Taskscited by 46
  13. 2019On Generalization Bounds of a Family of Recurrent Neural Networkscited by 42
  14. 2019Embedding and learning with signaturescited by 29
  15. 2016A scalable end-to-end Gaussian process adapter for irregularly sampled time series classificationcited by 27
  16. 2019TEASER: Early and Accurate Time Series Classificationcited by 26
  17. 2023Neural signature kernels as infinite-width-depth-limits of controlled ResNetscited by 25
  18. 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
  19. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  20. 2022Neural Stochastic Controlcited by 8
  21. 2022Fitting an immersed submanifold to data via Sussmann's orbit theoremcited by 5
  22. 2023Learning the Dynamics of Sparsely Observed Interacting Systemscited by 3
  23. 2023Benchmarking Continuous Time Models for Predicting Multiple Sclerosis Progressioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024A Generalization Bound for a Family of Implicit Networkscited by 4
  2. 2025Generalization Bound for a General Class of Neural Ordinary Differential Equationscited by 1
  3. 2026Constructive interpolation and generalization rates for neural ODEs: a control perspectivecited by 0