mlsanityData to 7 Oct 2026

Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

arXiv 2102.06559 ↗First posted 12 Feb 2021stat.ML cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

We perform scalable approximate inference in continuous-depth Bayesian neural networks. In this model class, uncertainty about separate weights in each layer gives hidden units that follow a stochastic differential equation. We demonstrate gradient-based stochastic variational inference in this infinite-parameter setting, producing arbitrarily-flexible approximate posteriors. We also derive a novel gradient estimator that approaches zero variance as the approximate posterior over weights approaches the true posterior. This approach brings continuous-depth Bayesian neural nets to a competitive comparison against discrete-depth alternatives, while inheriting the memory-efficient training and tunable precision of Neural ODEs.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 39 of its 60 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

39 of 60 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2018Neural Ordinary Differential Equationscited by 3,733
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2016HyperNetworkscited by 1,003
  9. 2015Weight Uncertainty in Neural Networkscited by 554
  10. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  11. 2019Augmented Neural ODEscited by 396
  12. 2017Stable Architectures for Deep Neural Networkscited by 340
27 more
  1. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  2. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  3. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 194
  4. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 182
  5. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  6. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  7. 2019Practical Deep Learning with Bayesian Principlescited by 158
  8. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  9. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 98
  10. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  11. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  12. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  13. 2020Learning Differential Equations that are Easy to Solvecited by 81
  14. 2020SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimatescited by 75
  15. 2017Bayesian Hypernetworkscited by 64
  16. 2015Bayesian Dark Knowledgecited by 45
  17. 2020Bayesian Neural Ordinary Differential Equationscited by 44
  18. 2019ODE2VAE: Deep generative second order ODEs with Bayesian neural networkscited by 36
  19. 2018SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradientcited by 34
  20. 2014Avoiding pathologies in very deep networkscited by 33
  21. 2018Adversarial Distillation of Bayesian Neural Network Posteriorscited by 27
  22. 2018Black-box Variational Inference for Stochastic Differential Equationscited by 24
  23. 2019Infinitely deep neural networks as diffusion processescited by 21
  24. 2020Stochasticity in Neural ODEs: An Empirical Studycited by 13
  25. 2019ANODEV2: A Coupled Neural ODE Evolution Frameworkcited by 12
  26. 2018Deep learning with differential Gaussian process flowscited by 11
  27. 2019Differential Bayesian Neural Netscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 7; 2023: 11; 2024: 6; 2025: 3; 2026: 3.

38 papers.

Sort:

By area:

  1. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  2. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  3. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  4. 2021Bayesian Learning via Neural Schr"odinger-F"ollmer Flowscited by 56
  5. 2021Constructing Neural Network-Based Models for Simulating Dynamical Systemscited by 29
  6. 2023Bayesian Neural Controlled Differential Equations for Treatment Effect Estimationcited by 26
  7. 2022Predicting the impact of treatments over time with uncertainty aware neural differential equationscited by 23
  8. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  9. 2022Learning Stochastic Dynamics with Statistics-Informed Neural Networkcited by 16
  10. 2023Thermodynamic AI and the fluctuation frontiercited by 15
  11. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  12. 2023Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equationscited by 13
26 more
  1. 2021GoTube: Scalable Stochastic Verification of Continuous-Depth Modelscited by 12
  2. 2022Variational Inference for Infinitely Deep Neural Networkscited by 10
  3. 2023How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equationscited by 10
  4. 2024Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equationscited by 8
  5. 2024Partially Stochastic Infinitely Deep Bayesian Neural Networkscited by 6
  6. 2024Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machinescited by 6
  7. 2023Learning nonlinear integral operators via Recurrent Neural Networks and its application in solving Integro-Differential Equationscited by 5
  8. 2023Non-Parametric Learning of Stochastic Differential Equations with Non-asymptotic Fast Rates of Convergencecited by 4
  9. 2024Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equationscited by 4
  10. 2023Faster Training of Neural ODEs Using Gauss-Legendre Quadraturecited by 3
  11. 2024Uncertainty Quantification on Graph Learning: A Surveycited by 3
  12. 2023Graph Neural Stochastic Differential Equationscited by 2
  13. 2025Uncertainty quantification of neural network models of evolving processes via Langevin samplingcited by 2
  14. 2021Distributional Gradient Matching for Learning Uncertain Neural Dynamics Modelscited by 1
  15. 2021Characteristic Neural Ordinary Differential Equationscited by 1
  16. 2022Realization Theory Of Recurrent Neural ODEs Using Polynomial System Embeddingscited by 1
  17. 2025HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Trainingcited by 1
  18. 2022Noisy Learning for Neural ODEs Acts as a Robustness Locus Wideningcited by 0
  19. 2022Continuous-time Particle Filtering for Latent Stochastic Differential Equationscited by 0
  20. 2022Changes from Classical Statistics to Modern Statistics and Data Sciencecited by 0
  21. 2023A Method to Improve the Performance of Reinforcement Learning Based on the Y Operator for a Class of Stochastic Differential Equation-Based Child-Mother Systemscited by 0
  22. 2024Improving the Noise Estimation of Latent Neural Stochastic Differential Equationscited by 0
  23. 2025Neural Stochastic Differential Equations on Compact State Spaces: Theory, Methods, and Application to Suicide Risk Modelingcited by 0
  24. 2026Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Methodcited by 0
  25. 2026Towards Intrinsically Calibrated Uncertainty Quantification in Industrial Data-Driven Models via Diffusion Samplercited by 0
  26. 2026Complex normalizing flows can almost be information K"ahler-Ricci flowscited by 0