mlsanityData to 14 Oct 2026

Kullback-Leibler excess risk bounds for exponential weighted aggregation in Generalized linear models

arXiv 2504.10171 ↗First posted 14 Apr 2025math.ST stat.ML stat.THcited by 0

Not on the map. Why?

Abstract (arXiv)

Aggregation methods have emerged as a powerful and flexible framework in statistical learning, providing unified solutions across diverse problems such as regression, classification, and density estimation. In the context of generalized linear models (GLMs), where responses follow exponential family distributions, aggregation offers an attractive alternative to classical parametric modeling. This paper investigates the problem of sparse aggregation in GLMs, aiming to approximate the true parameter vector by a sparse linear combination of predictors. We prove that an exponential weighted aggregation scheme yields a sharp oracle inequality for the Kullback-Leibler risk with leading constant equal to one, while also attaining the minimax-optimal rate of aggregation. These results are further enhanced by establishing high-probability bounds on the excess risk.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 44 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

14 of 44 references.

References · 14

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  2. 2019A Primer on PAC-Bayesian Learningcited by 132
  3. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  4. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  5. 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 61
  6. 2017High-dimensional classification by sparse logistic regressioncited by 19
  7. 2023A reduced-rank approach to predicting multiple binary responses through machine learningcited by 10
  8. 2023High-dimensional sparse classification using exponential weighting with empirical hinge losscited by 10
  9. 2024Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrixcited by 8
  10. 2014Model selection and minimax estimation in generalized linear modelscited by 7
  11. 2022From bilinear regression to inductive matrix completion: a quasi-Bayesian analysiscited by 7
  12. 2024Misclassification bounds for PAC-Bayesian sparse deep learningcited by 7
2 more
  1. 2024A sparse PAC-Bayesian approach for high-dimensional quantile predictioncited by 6
  2. 2025Optimal sparse phase retrieval via a quasi-Bayesian approachcited by 3

Built on by

Not cited yet.