mlsanityData to 7 Oct 2026

Model Selection in Bayesian Neural Networks via Horseshoe Priors

arXiv 1705.10388 ↗First posted 29 May 2017stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Bayesian Neural Networks (BNNs) have recently received increasing attention for their ability to provide well-calibrated posterior uncertainties. However, model selection---even choosing the number of nodes---remains an open question. In this work, we apply a horseshoe prior over node pre-activations of a Bayesian neural network, which effectively turns off nodes that do not help explain the data. We demonstrate that our prior prevents the BNN from under-fitting even when the number of nodes required is grossly over-estimated. Moreover, this model selection over the number of nodes doesn't come at the expense of predictive or computational performance; in fact, we learn smaller networks with comparable predictive performance to current approaches.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 14; 2023: 11; 2024: 5; 2025: 3; 2026: 7.

52 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021Bayesian Neural Network Priors Revisitedcited by 121
  3. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  4. 2023Credal Bayesian Deep Learningcited by 45
  5. 2022Post-hoc Uncertainty Learning using a Dirichlet Meta-Modelcited by 31
  6. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  7. 2021Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AIcited by 22
  8. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  9. 2022Beyond L1: Faster and Better Sparse Models with skglmcited by 16
  10. 2021Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Detailscited by 14
  11. 2022Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibilitycited by 13
  12. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
38 more
  1. 2022Variational Inference for Infinitely Deep Neural Networkscited by 10
  2. 2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
  3. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  4. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 9
  5. 2023Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metricscited by 8
  6. 2024Misclassification bounds for PAC-Bayesian sparse deep learningcited by 7
  7. 2022Sequential Bayesian Neural Subnetwork Ensemblescited by 5
  8. 2023Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertaintycited by 5
  9. 2023Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networkscited by 4
  10. 2021A fast asynchronous MCMC sampler for sparse Bayesian inferencecited by 3
  11. 2021Addressing Bias in Active Learning with Depth Uncertainty Networks... or Notcited by 3
  12. 2022Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networkscited by 3
  13. 2022Encoding Domain Knowledge in Multi-view Latent Variable Models: A Bayesian Approach with Structured Sparsitycited by 3
  14. 2022Bayesian Neural Networks for Macroeconomic Analysiscited by 3
  15. 2021Depth Uncertainty Networks for Active Learningcited by 2
  16. 2023Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representationscited by 2
  17. 2024BMRS: Bayesian Model Reduction for Structured Pruningcited by 2
  18. 2026Bayesian Lottery Ticket Hypothesiscited by 2
  19. 2021Can a single neuron learn predictive uncertainty?cited by 1
  20. 2022Masked Bayesian Neural Networks : Computation and Optimalitycited by 1
  21. 2022Interpretable (not just posthoc-explainable) medical claims modeling for discharge placement to prevent avoidable all-cause readmissions or deathcited by 1
  22. 2023Interpretable (not just posthoc-explainable) heterogeneous survivor bias-corrected treatment effects for assignment of postdischarge interventions to prevent readmissionscited by 1
  23. 2023Bayesian sparsification for deep neural networks with Bayesian model reductioncited by 1
  24. 2025Bayesian Computation in Deep Learningcited by 1
  25. 2025Vector Copula Variational Inference and Dependent Block Posterior Approximationscited by 1
  26. 2021Challenges and Opportunities in Approximate Bayesian Deep Learning for Intelligent IoT Systemscited by 0
  27. 2022Interpretable Latent Variables in Deep State Space Modelscited by 0
  28. 2022Radial Spike and Slab Bayesian Neural Networks for Sparse Data in Ransomware Attackscited by 0
  29. 2022A Bayesian Semiparametric Method For Estimating Causal Quantile Effectscited by 0
  30. 2023FineMorphs: Affine-diffeomorphic sequences for regressioncited by 0
  31. 2023Human Limits in Machine Learning: Prediction of Plant Phenotypes Using Soil Microbiome Datacited by 0
  32. 2024Semi-parametric Expert Bayesian Network Learning with Gaussian Processes and Horseshoe Priorscited by 0
  33. 2024Efficient Model Compression for Bayesian Neural Networkscited by 0
  34. 2025Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Priorcited by 0
  35. 2026Horseshoe Mixtures-of-Experts (HS-MoE)cited by 0
  36. 2026Bayesian inference with sources of uncertainty: from confidence modelling to sparse estimationcited by 0
  37. 2026Joint Model and Data Sparsification via the Marginal Likelihoodcited by 0
  38. 2026Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networkscited by 0

50 most cited of 52 All 52 (JSON)