# On the Hyperprior Choice for the Global Shrinkage Parameter in the Horseshoe Prior

arXiv: https://arxiv.org/abs/1610.05559 · HTML: https://mlsanity.com/paper/1610.05559/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2016.

## Built on by (most cited first, 10 of 10)

- [BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery](https://mlsanity.com/paper/2112.02761/) (2021)
- [Gaussian processes meet NeuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data](https://mlsanity.com/paper/2103.03385/) (2021)
- [FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference](https://mlsanity.com/paper/2502.02363/) (2025)
- [Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks](https://mlsanity.com/paper/2308.09104/) (2023)
- [T-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs](https://mlsanity.com/paper/2107.02510/) (2021)
- [Horseshoe Forests for High-Dimensional Causal Survival Analysis](https://mlsanity.com/paper/2507.22004/) (2025)
- [Autoencoded sparse Bayesian in-IRT factorization, calibration, and amortized inference for the Work Disability Functional Assessment Battery](https://mlsanity.com/paper/2210.10952/) (2022)
- [Semi-parametric Expert Bayesian Network Learning with Gaussian Processes and Horseshoe Priors](https://mlsanity.com/paper/2401.16419/) (2024)
- [Variational Bayesian Bow tie Neural Networks with Shrinkage](https://mlsanity.com/paper/2411.11132/) (2024)
- [Robust Bayes-Assisted Conformal Prediction](https://mlsanity.com/paper/2607.04236/) (2026)

## Abstract (arXiv)

The horseshoe prior has proven to be a noteworthy alternative for sparse Bayesian estimation, but as shown in this paper, the results can be sensitive to the prior choice for the global shrinkage hyperparameter. We argue that the previous default choices are dubious due to their tendency to favor solutions with more unshrunk coefficients than we typically expect a priori. This can lead to bad results if this parameter is not strongly identified by data. We derive the relationship between the global parameter and the effective number of nonzeros in the coefficient vector, and show an easy and intuitive way of setting up the prior for the global parameter based on our prior beliefs about the number of nonzero coefficients in the model. The results on real world data show that one can benefit greatly -- in terms of improved parameter estimates, prediction accuracy, and reduced computation time -- from transforming even a crude guess for the number of nonzero coefficients into the prior for the global parameter using our framework.