# Stochasticity in Neural ODEs: An Empirical Study

arXiv: https://arxiv.org/abs/2002.09779 · HTML: https://mlsanity.com/paper/2002.09779/

## Key facts

- Cited by 13 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 22 Feb 2020.

## Built on by (most cited first, 13 of 13)

- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations](https://mlsanity.com/paper/2102.06559/) (2021)
- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Beyond Predictions in Neural ODEs: Identification and Interventions](https://mlsanity.com/paper/2106.12430/) (2021)
- [Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2108.02318/) (2021)
- [Sparsity in Continuous-Depth Neural Networks](https://mlsanity.com/paper/2210.14672/) (2022)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations](https://mlsanity.com/paper/2410.10174/) (2024)
- [Meta-Solver for Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.08561/) (2021)
- [Continuum Dropout for Neural Differential Equations](https://mlsanity.com/paper/2511.10446/) (2025)
- [E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations](https://mlsanity.com/paper/2206.07578/) (2022)
- [Neural Controlled Differential Equations with Quantum Hidden Evolutions](https://mlsanity.com/paper/2404.19673/) (2024)
- [Modeling Neural Networks with Privacy Using Neural Stochastic Differential Equations](https://mlsanity.com/paper/2501.06686/) (2025)

## Abstract (arXiv)

Stochastic regularization of neural networks (e.g. dropout) is a wide-spread technique in deep learning that allows for better generalization. Despite its success, continuous-time models, such as neural ordinary differential equation (ODE), usually rely on a completely deterministic feed-forward operation. This work provides an empirical study of stochastically regularized neural ODE on several image-classification tasks (CIFAR-10, CIFAR-100, TinyImageNet). Building upon the formalism of stochastic differential equations (SDEs), we demonstrate that neural SDE is able to outperform its deterministic counterpart. Further, we show that data augmentation during the training improves the performance of both deterministic and stochastic versions of the same model. However, the improvements obtained by the data augmentation completely eliminate the empirical gains of the stochastic regularization, making the difference in the performance of neural ODE and neural SDE negligible.