# Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise

arXiv: https://arxiv.org/abs/1906.02355 · HTML: https://mlsanity.com/paper/1906.02355/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 30 of 101)

- [GRAND: Graph Neural Diffusion](https://mlsanity.com/paper/2106.10934/) (2021)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Beltrami Flow and Neural Diffusion on Graphs](https://mlsanity.com/paper/2110.09443/) (2021)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics](https://mlsanity.com/paper/2407.04192/) (2024)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics](https://mlsanity.com/paper/2110.10249/) (2021)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations](https://mlsanity.com/paper/2102.06559/) (2021)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Neural Delay Differential Equations](https://mlsanity.com/paper/2102.10801/) (2021)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Comprehensive Review of Neural Differential Equations for Time Series Analysis](https://mlsanity.com/paper/2502.09885/) (2025)
- [Policy Analysis using Synthetic Controls in Continuous-Time](https://mlsanity.com/paper/2102.01577/) (2021)
- [Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL](https://mlsanity.com/paper/2306.04220/) (2023)
- [TIDE: Time Derivative Diffusion for Deep Learning on Graphs](https://mlsanity.com/paper/2212.02483/) (2022)
- [Learning Stochastic Dynamics with Statistics-Informed Neural Network](https://mlsanity.com/paper/2202.12278/) (2022)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [From Coupled Oscillators to Graph Neural Networks: Reducing Over-smoothing via a Kuramoto Model-based Approach](https://mlsanity.com/paper/2311.03260/) (2023)
- [Neural network stochastic differential equation models with applications to financial data forecasting](https://mlsanity.com/paper/2111.13164/) (2021)
- [Improving Transferability for Cross-domain Trajectory Prediction via Neural Stochastic Differential Equation](https://mlsanity.com/paper/2312.15906/) (2023)
- [Fractional SDE-Net: Generation of Time Series Data with Long-term Memory](https://mlsanity.com/paper/2201.05974/) (2022)
- [Variational Inference for SDEs Driven by Fractional Noise](https://mlsanity.com/paper/2310.12975/) (2023)
- [From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems](https://mlsanity.com/paper/2405.11542/) (2024)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Path Development Network with Finite-dimensional Lie Group Representation](https://mlsanity.com/paper/2204.00740/) (2022)
- [Learning Differential Operators for Interpretable Time Series Modeling](https://mlsanity.com/paper/2209.01491/) (2022)
- [In-Context Learning of Stochastic Differential Equations with Foundation Inference Models](https://mlsanity.com/paper/2502.19049/) (2025)
- [Partially Stochastic Infinitely Deep Bayesian Neural Networks](https://mlsanity.com/paper/2402.03495/) (2024)

## Abstract (arXiv)

Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous neural network framework called Neural Stochastic Differential Equation (Neural SDE) network, which naturally incorporates various commonly used regularization mechanisms based on random noise injection. Our framework can model various types of noise injection frequently used in discrete networks for regularization purpose, such as dropout and additive/multiplicative noise in each block. We provide theoretical analysis explaining the improved robustness of Neural SDE models against input perturbations/adversarial attacks. Furthermore, we demonstrate that the Neural SDE network can achieve better generalization than the Neural ODE and is more resistant to adversarial and non-adversarial input perturbations.