mlsanityData to 7 Oct 2026

Standalone Neural ODEs with Sensitivity Analysis

arXiv 2205.13933 ↗First posted 27 May 2022cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents the Standalone Neural ODE (sNODE), a continuous-depth neural ODE model capable of describing a full deep neural network. This uses a novel nonlinear conjugate gradient (NCG) descent optimization scheme for training, where the Sobolev gradient can be incorporated to improve smoothness of model weights. We also present a general formulation of the neural sensitivity problem and show how it is used in the NCG training. The sensitivity analysis provides a reliable measure of uncertainty propagation throughout a network, and can be used to study model robustness and to generate adversarial attacks. Our evaluations demonstrate that our novel formulations lead to increased robustness and performance as compared to ResNet models, and that it opens up for new opportunities for designing and developing machine learning with improved explainability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 27 May 2022.

Builds on

23 of 28 references.

References · 23

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  9. 2019Augmented Neural ODEscited by 396
  10. 2017Stable Architectures for Deep Neural Networkscited by 340
  11. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  12. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
11 more
  1. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
  2. 2020Dissecting Neural ODEscited by 150
  3. 2020Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Taskscited by 137
  4. 2017Sobolev Training for Neural Networkscited by 136
  5. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 98
  6. 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
  7. 2019On Robustness of Neural Ordinary Differential Equationscited by 83
  8. 2020NeuralSens: Sensitivity Analysis of Neural Networkscited by 14
  9. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  10. 2021Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memorycited by 11
  11. 2022Learning via nonlinear conjugate gradients and depth-varying neural ODEscited by 3

Built on by

Not cited yet.