# Aggregation of Affine Estimators

arXiv: https://arxiv.org/abs/1311.2799 · HTML: https://mlsanity.com/paper/1311.2799/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2013.

## Built on by (most cited first, 4 of 4)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Mallows-type model averaging: Non-asymptotic analysis and all-subset combination](https://mlsanity.com/paper/2505.02637/) (2025)
- [Hedging Memory Horizons for Non-Stationary Prediction via Online Aggregation](https://mlsanity.com/paper/2605.06541/) (2026)
- [Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting](https://mlsanity.com/paper/2607.22299/) (2026)

## Abstract (arXiv)

We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon (2012) have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced Q-aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation and show that our proposed estimator leads simultaneously to all the best known bounds for aggregation, including ℓq-aggregation, q ∈ (0,1), with high probability.