# High-dimensional prediction for count response via sparse exponential weights

arXiv: https://arxiv.org/abs/2410.15381 · HTML: https://mlsanity.com/paper/2410.15381/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 75 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Bayesian linear regression with sparse priors](https://mlsanity.com/paper/1403.0735/) (2014)
- [High-dimensional classification by sparse logistic regression](https://mlsanity.com/paper/1706.08344/) (2017)
- [A reduced-rank approach to predicting multiple binary responses through machine learning](https://mlsanity.com/paper/2306.05829/) (2023)
- [High-dimensional sparse classification using exponential weighting with empirical hinge loss](https://mlsanity.com/paper/2312.12952/) (2023)
- [Elastic-net Regularized High-dimensional Negative Binomial Regression: Consistency and Weak Signals Detection](https://mlsanity.com/paper/1712.03412/) (2017)
- [Model selection and minimax estimation in generalized linear models](https://mlsanity.com/paper/1409.8491/) (2014)
- [From bilinear regression to inductive matrix completion: a quasi-Bayesian analysis](https://mlsanity.com/paper/2210.15290/) (2022)
- [Misclassification bounds for PAC-Bayesian sparse deep learning](https://mlsanity.com/paper/2405.01304/) (2024)
- [A sparse PAC-Bayesian approach for high-dimensional quantile prediction](https://mlsanity.com/paper/2409.01687/) (2024)
- [PAC-Bayesian AUC classification and scoring](https://mlsanity.com/paper/1410.1771/) (2014)
- [PAC-Bayesian High Dimensional Bipartite Ranking](https://mlsanity.com/paper/1511.02729/) (2015)
- [Heterogeneous Overdispersed Count Data Regressions via Double Penalized Estimations](https://mlsanity.com/paper/2110.03552/) (2021)
- [Variational Inference for Sparse Poisson Regression](https://mlsanity.com/paper/2311.01147/) (2023)
- [High-dimensional regression with a count response](https://mlsanity.com/paper/2409.08821/) (2024)

## Abstract (arXiv)

Count data is prevalent in various fields like ecology, medical research, and genomics. In high-dimensional settings, where the number of features exceeds the sample size, feature selection becomes essential. While frequentist methods like Lasso have advanced in handling high-dimensional count data, Bayesian approaches remain under-explored with no theoretical results on prediction performance. This paper introduces a novel probabilistic machine learning framework for high-dimensional count data prediction. We propose a pseudo-Bayesian method that integrates a scaled Student prior to promote sparsity and uses an exponential weight aggregation procedure. A key contribution is a novel risk measure tailored to count data prediction, with theoretical guarantees for prediction risk using PAC-Bayesian bounds. Our results include non-asymptotic oracle inequalities, demonstrating rate-optimal prediction error without prior knowledge of sparsity. We implement this approach efficiently using Langevin Monte Carlo method. Simulations and a real data application highlight the strong performance of our method compared to the Lasso in various settings.