# Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

arXiv: https://arxiv.org/abs/2402.14989 · HTML: https://mlsanity.com/paper/2402.14989/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 75 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Doctor AI: Predicting Clinical Events via Recurrent Neural Networks](https://mlsanity.com/paper/1511.05942/) (2015)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [The Effect of Natural Distribution Shift on Question Answering Models](https://mlsanity.com/paper/2004.14444/) (2020)
- [sktime: A Unified Interface for Machine Learning with Time Series](https://mlsanity.com/paper/1909.07872/) (2019)
- [Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise](https://mlsanity.com/paper/1906.02355/) (2019)

## Built on by (most cited first, 30 of 36)

- [Trajectory Flow Matching with Applications to Clinical Time Series Modeling](https://mlsanity.com/paper/2410.21154/) (2024)
- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Comprehensive Review of Neural Differential Equations for Time Series Analysis](https://mlsanity.com/paper/2502.09885/) (2025)
- [ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images](https://mlsanity.com/paper/2406.14794/) (2024)
- [Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching](https://mlsanity.com/paper/2405.20799/) (2024)
- [Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs](https://mlsanity.com/paper/2601.11061/) (2026)
- [Feedback Favors the Generalization of Neural ODEs](https://mlsanity.com/paper/2410.10253/) (2024)
- [LSCD: Lomb-Scargle Conditioned Diffusion for Time series Imputation](https://mlsanity.com/paper/2506.17039/) (2025)
- [Strong Stochastic Flow Maps](https://mlsanity.com/paper/2606.01086/) (2026)
- [DeNOTS: Stable Deep Neural ODEs for Time Series](https://mlsanity.com/paper/2408.08055/) (2024)
- [Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions](https://mlsanity.com/paper/2510.25769/) (2025)
- [Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equations](https://mlsanity.com/paper/2412.12112/) (2024)
- [Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis](https://mlsanity.com/paper/2506.22393/) (2025)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [ODEBrain: Continuous-Time EEG Graph for Modeling Dynamic Brain Networks](https://mlsanity.com/paper/2602.23285/) (2026)
- [Neural Stochastic Processes for Satellite Precipitation Refinement](https://mlsanity.com/paper/2604.10414/) (2026)
- [TRENDy: Temporal Regression of Effective Nonlinear Dynamics](https://mlsanity.com/paper/2412.03496/) (2024)
- [multivariateGPT: a decoder-only transformer for multivariate categorical and numeric data](https://mlsanity.com/paper/2505.21680/) (2025)
- [Uncertainty Propagation Networks for Neural Ordinary Differential Equations](https://mlsanity.com/paper/2508.16815/) (2025)
- [Generalization Bound for a General Class of Neural Ordinary Differential Equations](https://mlsanity.com/paper/2508.18920/) (2025)
- [Explicit and Effectively Symmetric Schemes for Neural SDEs on Lie Groups](https://mlsanity.com/paper/2509.20599/) (2025)
- [SDE-Attention: Latent Attention in SDE-RNNs for Irregularly Sampled Time Series with Missing Data](https://mlsanity.com/paper/2511.23238/) (2025)
- [Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations](https://mlsanity.com/paper/2603.18331/) (2026)
- [The Stochastic Occupation Kernel (SOCK) Method for Learning Stochastic Differential Equations](https://mlsanity.com/paper/2505.11622/) (2025)
- [Neural Stochastic Differential Equations on Compact State Spaces: Theory, Methods, and Application to Suicide Risk Modeling](https://mlsanity.com/paper/2508.17090/) (2025)
- [Learning Aligned Stability in Neural ODEs Reconciling Accuracy with Robustness](https://mlsanity.com/paper/2509.21879/) (2025)
- [FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression](https://mlsanity.com/paper/2510.00621/) (2025)
- [FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correction](https://mlsanity.com/paper/2510.27173/) (2025)
- [Data-driven stochastic reduced-order modeling of parametrized dynamical systems](https://mlsanity.com/paper/2601.10690/) (2026)
- [CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning](https://mlsanity.com/paper/2602.09207/) (2026)

## Abstract (arXiv)

Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an alternative approach, utilizing neural networks combined with ODE solvers to learn continuous latent representations through parameterized vector fields. Neural Stochastic Differential Equations (Neural SDEs) extend Neural ODEs by incorporating a diffusion term, although this addition is not trivial, particularly when addressing irregular intervals and missing values. Consequently, careful design of drift and diffusion functions is crucial for maintaining stability and enhancing performance, while incautious choices can result in adverse properties such as the absence of strong solutions, stochastic destabilization, or unstable Euler discretizations, significantly affecting Neural SDEs' performance. In this study, we propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Then, we rigorously demonstrate their robustness in maintaining excellent performance under distribution shift, while effectively preventing overfitting. To assess the effectiveness of our approach, we conduct extensive experiments on four benchmark datasets for interpolation, forecasting, and classification tasks, and analyze the robustness of our methods with 30 public datasets under different missing rates. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.