# A sparse PAC-Bayesian approach for high-dimensional quantile prediction

arXiv: https://arxiv.org/abs/2409.01687 · HTML: https://mlsanity.com/paper/2409.01687/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 86 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm](https://mlsanity.com/paper/1503.08329/) (2015)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Bayesian linear regression with sparse priors](https://mlsanity.com/paper/1403.0735/) (2014)
- [General Bayesian Updating and the Loss-Likelihood Bootstrap](https://mlsanity.com/paper/1709.07616/) (2017)
- [Calibrating general posterior credible regions](https://mlsanity.com/paper/1509.00922/) (2015)
- [abess: A Fast Best Subset Selection Library in Python and R](https://mlsanity.com/paper/2110.09697/) (2021)
- [On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations](https://mlsanity.com/paper/2104.08324/) (2021)
- [Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression](https://mlsanity.com/paper/1509.02957/) (2015)
- [General Bayesian Loss Function Selection and the use of Improper Models](https://mlsanity.com/paper/2106.01214/) (2021)
- [High-dimensional sparse classification using exponential weighting with empirical hinge loss](https://mlsanity.com/paper/2312.12952/) (2023)
- [Adaptation of the Tuning Parameter in General Bayesian Inference with Robust Divergence](https://mlsanity.com/paper/2106.06902/) (2021)
- [Horseshoe Prior Bayesian Quantile Regression](https://mlsanity.com/paper/2006.07655/) (2020)
- [A Sandwich Likelihood Correction for Bayesian Quantile Regression based on the Misspecified Asymmetric Laplace Density](https://mlsanity.com/paper/1502.06481/) (2015)
- [Variable Selection for Additive Partial Linear Quantile Regression with Missing Covariates](https://mlsanity.com/paper/1510.00094/) (2015)
- [Joint estimation of conditional quantiles in multivariate linear regression models. An application to financial distress](https://mlsanity.com/paper/1808.01240/) (2018)
- [Sparse Quantile Regression](https://mlsanity.com/paper/2006.11201/) (2020)

## Built on by (most cited first, 6 of 6)

- [Optimal sparse phase retrieval via a quasi-Bayesian approach](https://mlsanity.com/paper/2504.09509/) (2025)
- [High-dimensional prediction for count response via sparse exponential weights](https://mlsanity.com/paper/2410.15381/) (2024)
- [Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models](https://mlsanity.com/paper/2504.10171/) (2025)
- [Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss](https://mlsanity.com/paper/2511.15332/) (2025)
- [Sparse classification with positive-confidence data in high dimensions](https://mlsanity.com/paper/2512.24443/) (2025)
- [Robust low-rank estimation with multiple binary responses using pairwise AUC loss](https://mlsanity.com/paper/2601.08618/) (2026)

## Abstract (arXiv)

Quantile regression, a robust method for estimating conditional quantiles, has advanced significantly in fields such as econometrics, statistics, and machine learning. In high-dimensional settings, where the number of covariates exceeds sample size, penalized methods like lasso have been developed to address sparsity challenges. Bayesian methods, initially connected to quantile regression via the asymmetric Laplace likelihood, have also evolved, though issues with posterior variance have led to new approaches, including pseudo/score likelihoods. This paper presents a novel probabilistic machine learning approach for high-dimensional quantile prediction. It uses a pseudo-Bayesian framework with a scaled Student-t prior and Langevin Monte Carlo for efficient computation. The method demonstrates strong theoretical guarantees, through PAC-Bayes bounds, that establish non-asymptotic oracle inequalities, showing minimax-optimal prediction error and adaptability to unknown sparsity. Its effectiveness is validated through simulations and real-world data, where it performs competitively against established frequentist and Bayesian techniques.