mlsanityData to 7 Oct 2026

Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method

arXiv 2603.25024 ↗First posted 26 Mar 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

As a representative continuous-depth neural network approach, stochastic differential equation (SDE)-based Bayesian neural networks (BNNs) have attracted considerable attention due to their solid theoretical foundations and strong potential for real-world applications. However, their reliance on numerical SDE solvers inevitably incurs a large number of function evaluations (NFEs), resulting in high computational cost and occasional convergence instability. To address these challenges, we propose a Nesterov-accelerated gradient (NAG) enhanced SDE-BNN model. By integrating NAG into the SDE-BNN framework along with an NFE-dependent residual skip connection, our method accelerates convergence and substantially reduces NFEs during both training and testing. Extensive empirical results show that our model consistently outperforms conventional SDE-BNNs across various tasks, including image classification and sequence modeling, achieving lower NFEs and improved predictive accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

20 of 33 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  5. 2019Augmented Neural ODEscited by 396
  6. 2020Lagrangian Neural Networkscited by 388
  7. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  8. 2019Neural Jump Stochastic Differential Equationscited by 147
  9. 2020Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraintscited by 102
  10. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 98
  11. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 82
  12. 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
8 more
  1. 2019Machine Learning from a Continuous Viewpointcited by 50
  2. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  3. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  4. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  5. 2020Neural Ordinary Differential Equation Control of Dynamics on Graphscited by 24
  6. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  7. 2023Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equationscited by 13
  8. 2024Partially Stochastic Infinitely Deep Bayesian Neural Networkscited by 6

Built on by

Not cited yet.