mlsanityData to 7 Oct 2026

Meta-strategy for Learning Tuning Parameters with Guarantees

arXiv 2102.02504 ↗First posted 4 Feb 2021stat.ML cs.LG math.STcited by 6

Not on the map. Why?

Abstract (arXiv)

Online learning methods, like the online gradient algorithm (OGA) and exponentially weighted aggregation (EWA), often depend on tuning parameters that are difficult to set in practice. We consider an online meta-learning scenario, and we propose a meta-strategy to learn these parameters from past tasks. Our strategy is based on the minimization of a regret bound. It allows to learn the initialization and the step size in OGA with guarantees. It also allows to learn the prior or the learning rate in EWA. We provide a regret analysis of the strategy. It allows to identify settings where meta-learning indeed improves on learning each task in isolation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 4 Feb 2021.

Builds on

23 of 45 references.

References · 23

  1. 2019Introduction to Online Convex Optimizationcited by 793
  2. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  3. 2016Learning to learn by gradient descent by gradient descentcited by 616
  4. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  5. 2015The Benefit of Multitask Representation Learningcited by 194
  6. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  7. 2019Online Meta-Learningcited by 137
  8. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  9. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  10. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  11. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  12. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
11 more
  1. 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
  2. 2019Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximationscited by 51
  3. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  4. 2020The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuningcited by 40
  5. 2019Provable Smoothness Guarantees for Black-Box Variational Inferencecited by 29
  6. 2020Temporal Variability in Implicit Online Learningcited by 20
  7. 2016Regret Bounds for Lifelong Learningcited by 19
  8. 2020Non-exponentially weighted aggregation: regret bounds for unbounded loss functionscited by 18
  9. 2019A Generalization Bound for Online Variational Inferencecited by 17
  10. 2020Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimizationcited by 14
  11. 2016Localized Lasso for High-Dimensional Regressioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 0; 2025: 0; 2026: 1.

6 papers.

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  3. 2023Nonlinear Meta-Learning Can Guarantee Faster Ratescited by 8
  4. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  5. 2022Online Meta-Learning in Adversarial Multi-Armed Banditscited by 2
  6. 2026Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Banditscited by 0