mlsanityData to 7 Oct 2026

Non-exponentially weighted aggregation: regret bounds for unbounded loss functions

arXiv 2009.03017 ↗First posted 7 Sep 2020stat.ML cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

We tackle the problem of online optimization with a general, possibly unbounded, loss function. It is well known that when the loss is bounded, the exponentially weighted aggregation strategy (EWA) leads to a regret in √T after T steps. In this paper, we study a generalized aggregation strategy, where the weights no longer depend exponentially on the losses. Our strategy is based on Follow The Regularized Leader (FTRL): we minimize the expected losses plus a regularizer, that is here a φ-divergence. When the regularizer is the Kullback-Leibler divergence, we obtain EWA as a special case. Using alternative divergences enables unbounded losses, at the cost of a worst regret bound in some cases.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 1; 2023: 4; 2024: 2; 2025: 4; 2026: 4.

18 papers.

Sort:

  1. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 31
  2. 2021On the Robustness to Misspecification of α-Posteriors and Their Variational Approximationscited by 15
  3. 2023Online-to-PAC Conversions: Generalization Bounds via Regret Analysiscited by 14
  4. 2023Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?cited by 10
  5. 2022Optimal Comparator Adaptive Online Learning with Switching Costcited by 9
  6. 2021Minimax Optimal Quantile and Semi-Adversarial Regret via Root-Logarithmic Regularizerscited by 8
  7. 2021Meta-strategy for Learning Tuning Parameters with Guaranteescited by 6
  8. 2023Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inferencecited by 6
  9. 2024Equivalence of the Empirical Risk Minimization to Regularization on the Family of f-Divergencescited by 6
  10. 2024Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimizationcited by 6
  11. 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Frameworkcited by 1
  12. 2025Fenchel-Young Variational Learningcited by 1
6 more
  1. 2025A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularizationcited by 1
  2. 2026Adaptive Bayesian Online Learning via Expert Aggregationcited by 1
  3. 2025Bayesian Algorithms for Adversarial Online Learning: from Finite to Infinite Action Spacescited by 0
  4. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0
  5. 2026Hedging Memory Horizons for Non-Stationary Prediction via Online Aggregationcited by 0
  6. 2026Fast rates in Bayesian online learning with approximate posteriorscited by 0