Local Risk Bounds for Statistical Aggregation
Not on the map. Why?
Abstract (arXiv)
In the problem of aggregation, the aim is to combine a given class of base predictors to achieve predictions nearly as accurate as the best one. In this flexible framework, no assumption is made on the structure of the class or the nature of the target. Aggregation has been studied in both sequential and statistical contexts. Despite some important differences between the two problems, the classical results in both cases feature the same global complexity measure. In this paper, we revisit and tighten classical results in the theory of aggregation in the statistical setting by replacing the global complexity with a smaller, local one. Some of our proofs build on the PAC-Bayes localization technique introduced by Catoni. Among other results, we prove localized versions of the classical bound for the exponential weights estimator due to Leung and Barron and deviation-optimal bounds for the Q-aggregation estimator. These bounds improve over the results of Dai, Rigollet and Zhang for fixed design regression and the results of Lecu'e and Rigollet for random design regression.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 77 references that are in the map. First posted on arXiv 29 Jun 2023.
Builds on
23 of 77 references.
References · 23
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
- 2021User-friendly introduction to PAC-Bayes boundscited by 234
- 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
- 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
- 2014Minimax Analysis of Active Learningcited by 61
- 2013Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributionscited by 59
- 2015Second-order Quantile Methods for Experts and Combinatorial Gamescited by 47
- 2015Learning with Square Loss: Localization through Offset Rademacher Complexitycited by 37
- 2014Optimal learning with Bernstein Online Aggregationcited by 35
- 2021PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classescited by 31
- 2021Towards a Unified Information-Theoretic Framework for Generalizationcited by 29
- 2021MetaGrad: Adaptation using Multiple Learning Rates in Online Learningcited by 26
11 more
- 2015Refined Error Bounds for Several Learning Algorithmscited by 24
- 2019An improper estimator with optimal excess risk in misspecified density estimation and logistic regressioncited by 24
- 2013On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functionscited by 23
- 2013Empirical entropy, minimax regret and minimax riskcited by 21
- 2021Distribution-Free Robust Linear Regressioncited by 19
- 2014The Sample Complexity of Learning Linear Predictors with the Squared Losscited by 16
- 2023Online-to-PAC Conversions: Generalization Bounds via Regret Analysiscited by 14
- 2022Exponential Tail Local Rademacher Complexity Risk Bounds Without the Bernstein Conditioncited by 9
- 2015On aggregation for heavy-tailed classescited by 7
- 2021Localization, Convexity, and Star Aggregationcited by 7
- 2022A Regret-Variance Trade-Off in Online Learningcited by 7
Built on by
Citing papers per year: 2026: 3.
3 papers.