mlsanityData to 7 Oct 2026

Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference

arXiv 2305.14765 ↗First posted 24 May 2023stat.ML cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Bayesian approaches for learning deep neural networks (BNN) have been received much attention and successfully applied to various applications. Particularly, BNNs have the merit of having better generalization ability as well as better uncertainty quantification. For the success of BNN, search an appropriate architecture of the neural networks is an important task, and various algorithms to find good sparse neural networks have been proposed. In this paper, we propose a new node-sparse BNN model which has good theoretical properties and is computationally feasible. We prove that the posterior concentration rate to the true model is near minimax optimal and adaptive to the smoothness of the true model. In particular the adaptiveness is the first of its kind for node-sparse BNNs. In addition, we develop a novel MCMC algorithm which makes the Bayesian inference of the node-sparse BNN model feasible in practice.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 43 of its 75 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

43 of 75 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  8. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  9. 2016Error bounds for approximations with deep ReLU networkscited by 567
  10. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
31 more
  1. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  2. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  4. 2019Verified Uncertainty Calibrationcited by 307
  5. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  6. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  7. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  8. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  9. 2017Bayesian Compression for Deep Learningcited by 116
  10. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  11. 2014Collaborative Deep Learning for Recommender Systemscited by 95
  12. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  13. 2019Towards Efficient Model Compression via Learned Global Rankingcited by 75
  14. 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
  15. 2017Model Selection in Bayesian Neural Networks via Horseshoe Priorscited by 52
  16. 2018Posterior Concentration for Sparse Deep Learningcited by 50
  17. 2022A Langevin-like Sampler for Discrete Distributionscited by 42
  18. 2014Bayesian linear regression with sparse priorscited by 40
  19. 2019Learning Sparse Networks Using Targeted Dropoutcited by 40
  20. 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
  21. 2021Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filteringcited by 31
  22. 2019Convergence Rates of Variational Inference in Sparse Deep Learningcited by 27
  23. 2019An Adaptive Empirical Bayesian Method for Sparse Deep Learningcited by 20
  24. 2021A Bayesian neural network predicts the dissolution of compact planetary systemscited by 15
  25. 2020Asymptotically Optimal Exact Minibatch Metropolis-Hastingscited by 14
  26. 2021Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Detailscited by 14
  27. 2018Multivariate Bayesian Structural Time Series Modelcited by 11
  28. 2022Asymptotic Properties for Bayesian Neural Network in Besov Spacecited by 7
  29. 2019Mini-batch Metropolis-Hastings MCMC with Reversible SGLD Proposalcited by 6
  30. 2020Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applicationscited by 6
  31. 2020Neural Bootstrappercited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 2.

9 papers.

  1. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 9
  2. 2024Training Bayesian Neural Networks with Sparse Subspace Variational Inferencecited by 7
  3. 2024Misclassification bounds for PAC-Bayesian sparse deep learningcited by 7
  4. 2025Stochastic Weight Sharing for Bayesian Neural Networkscited by 2
  5. 2025Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitycited by 2
  6. 2025Knowledge Distillation of Uncertainty using Deep Latent Factor Modelcited by 1
  7. 2025Bayesian Neural Networks for Functional ANOVA modelcited by 0
  8. 2026Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEscited by 0
  9. 2026Rate-optimal neural boundary detection from unlabeled noisy imagescited by 0