mlsanityData to 13 Oct 2026

Scalable Bayesian Deep Learning with Kernel Seed Networks

arXiv 2104.09005 ↗First posted 19 Apr 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper addresses the scalability problem of Bayesian deep neural networks. The performance of deep neural networks is undermined by the fact that these algorithms have poorly calibrated measures of uncertainty. This restricts their application in high risk domains such as computer aided diagnosis and autonomous vehicle navigation. Bayesian Deep Learning (BDL) offers a promising method for representing uncertainty in neural network. However, BDL requires a separate set of parameters to store the mean and standard deviation of model weights to learn a distribution. This results in a prohibitive 2-fold increase in the number of model parameters. To address this problem we present a method for performing BDL, namely Kernel Seed Networks (KSN), which does not require a 2-fold increase in the number of parameters. KSNs use 1x1 Convolution operations to learn a compressed latent space representation of the parameter distribution. In this paper we show how this allows KSNs to outperform conventional BDL methods while reducing the number of required parameters by up to a factor of 6.6.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

15 of 24 references.

References · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,321
  4. 2017On Calibration of Modern Neural Networkscited by 4,902
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,430
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,328
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,902
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,601
  10. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,340
  11. 2015Weight Uncertainty in Neural Networkscited by 554
  12. 2017Variational Continual Learningcited by 335
3 more
  1. 2019Practical Deep Learning with Bayesian Principlescited by 158
  2. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  3. 2019A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inferencecited by 51

Built on by

Not cited yet.