# Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models

arXiv: https://arxiv.org/abs/2504.10171 · HTML: https://mlsanity.com/paper/2504.10171/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 14 of 14)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Generalization Bounds: Perspectives from Information Theory and PAC-Bayes](https://mlsanity.com/paper/2309.04381/) (2023)
- [High-dimensional classification by sparse logistic regression](https://mlsanity.com/paper/1706.08344/) (2017)
- [A reduced-rank approach to predicting multiple binary responses through machine learning](https://mlsanity.com/paper/2306.05829/) (2023)
- [High-dimensional sparse classification using exponential weighting with empirical hinge loss](https://mlsanity.com/paper/2312.12952/) (2023)
- [Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix](https://mlsanity.com/paper/2406.14269/) (2024)
- [Model selection and minimax estimation in generalized linear models](https://mlsanity.com/paper/1409.8491/) (2014)
- [From bilinear regression to inductive matrix completion: a quasi-Bayesian analysis](https://mlsanity.com/paper/2210.15290/) (2022)
- [Misclassification bounds for PAC-Bayesian sparse deep learning](https://mlsanity.com/paper/2405.01304/) (2024)
- [A sparse PAC-Bayesian approach for high-dimensional quantile prediction](https://mlsanity.com/paper/2409.01687/) (2024)
- [Optimal sparse phase retrieval via a quasi-Bayesian approach](https://mlsanity.com/paper/2504.09509/) (2025)

## Abstract (arXiv)

Aggregation methods have emerged as a powerful and flexible framework in statistical learning, providing unified solutions across diverse problems such as regression, classification, and density estimation. In the context of generalized linear models (GLMs), where responses follow exponential family distributions, aggregation offers an attractive alternative to classical parametric modeling. This paper investigates the problem of sparse aggregation in GLMs, aiming to approximate the true parameter vector by a sparse linear combination of predictors. We prove that an exponential weighted aggregation scheme yields a sharp oracle inequality for the Kullback-Leibler risk with leading constant equal to one, while also attaining the minimax-optimal rate of aggregation. These results are further enhanced by establishing high-probability bounds on the excess risk.