# Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equations

arXiv: https://arxiv.org/abs/2112.12707 · HTML: https://mlsanity.com/paper/2112.12707/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 23 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs](https://mlsanity.com/paper/1902.10298/) (2019)
- [On Robustness of Neural Ordinary Differential Equations](https://mlsanity.com/paper/1910.05513/) (2019)
- [MIT Advanced Vehicle Technology Study: Large-Scale Naturalistic Driving Study of Driver Behavior and Interaction with Automation](https://mlsanity.com/paper/1711.06976/) (2017)
- [Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic data](https://mlsanity.com/paper/1812.08733/) (2018)

## Built on by (most cited first, 7 of 7)

- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Uncertainty and Structure in Neural Ordinary Differential Equations](https://mlsanity.com/paper/2305.13290/) (2023)
- [Learning Subgrid-scale Models with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2212.09967/) (2022)
- [Zero Stability Well Predicts Performance of Convolutional Neural Networks](https://mlsanity.com/paper/2206.13100/) (2022)
- [Deep Learning Aided Laplace Based Bayesian Inference for Epidemiological Systems](https://mlsanity.com/paper/2210.08865/) (2022)
- [Differentiable DG with Neural Operator Source Term Correction](https://mlsanity.com/paper/2310.18897/) (2023)
- [Modeling Neural Networks with Privacy Using Neural Stochastic Differential Equations](https://mlsanity.com/paper/2501.06686/) (2025)

## Abstract (arXiv)

Neural ordinary differential equations (NODE) have been proposed as a continuous depth generalization to popular deep learning models such as Residual networks (ResNets). They provide parameter efficiency and automate the model selection process in deep learning models to some extent. However, they lack the much-required uncertainty modelling and robustness capabilities which are crucial for their use in several real-world applications such as autonomous driving and healthcare. We propose a novel and unique approach to model uncertainty in NODE by considering a distribution over the end-time T of the ODE solver. The proposed approach, latent time NODE (LT-NODE), treats T as a latent variable and apply Bayesian learning to obtain a posterior distribution over T from the data. In particular, we use variational inference to learn an approximate posterior and the model parameters. Prediction is done by considering the NODE representations from different samples of the posterior and can be done efficiently using a single forward pass. As T implicitly defines the depth of a NODE, posterior distribution over T would also help in model selection in NODE. We also propose, adaptive latent time NODE (ALT-NODE), which allow each data point to have a distinct posterior distribution over end-times. ALT-NODE uses amortized variational inference to learn an approximate posterior using inference networks. We demonstrate the effectiveness of the proposed approaches in modelling uncertainty and robustness through experiments on synthetic and several real-world image classification data.