mlsanityData to 14 Oct 2026

Meta-Learning Reliable Priors in the Function Space

arXiv 2106.03195 ↗First posted 6 Jun 2021cs.LG cs.AI stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks. However, existing methods have unreliable uncertainty estimates which are often overconfident. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level regularization directly in the function space. This allows us to directly steer the probabilistic predictions of the meta-learner towards high epistemic uncertainty in regions of insufficient meta-training data and, thus, obtain well-calibrated uncertainty estimates. Finally, we showcase how our approach can be integrated with sequential decision making, where reliable uncertainty quantification is imperative. In our benchmark study on meta-learning for Bayesian Optimization (BO), F-PACOH significantly outperforms all other meta-learners and standard baselines.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 35 of its 79 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

35 of 79 references.

References · 35

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  4. 2016Matching Networks for One Shot Learningcited by 2,255
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  6. 2018A Tutorial on Bayesian Optimizationcited by 744
  7. 2016Learning to learn by gradient descent by gradient descentcited by 620
  8. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
  9. 2015Deep Kernel Learningcited by 445
  10. 2018Conditional Neural Processescited by 426
  11. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  12. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
23 more
  1. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  2. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 206
  3. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  4. 2019A Primer on PAC-Bayesian Learningcited by 132
  5. 2019Functional Variational Bayesian Neural Networkscited by 127
  6. 2019Meta-Learning without Memorizationcited by 92
  7. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  8. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  9. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  10. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  11. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  12. 2015On Sparse variational methods and the Kullback-Leibler divergence between stochastic processescited by 57
  13. 2018Meta-Learning Priors for Efficient Online Bayesian Regressioncited by 48
  14. 2018A Spectral Approach to Gradient Estimation for Implicit Distributionscited by 45
  15. 2018Variational Implicit Processescited by 41
  16. 2020Understanding Variational Inference in Function-Spacecited by 40
  17. 2019Noise Regularization for Conditional Density Estimationcited by 23
  18. 2017Warmstarting of Model-based Algorithm Configurationcited by 17
  19. 2013On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoodscited by 16
  20. 2020SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Modelscited by 14
  21. 2018Automatic Exploration of Machine Learning Experiments on OpenMLcited by 13
  22. 2018Hyperparameter Learning via Distributional Transfercited by 13
  23. 2019Transferable Neural Processes for Hyperparameter Optimizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 7; 2023: 6; 2024: 1; 2025: 1; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 92
  2. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
  3. 2022Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Predictioncited by 17
  4. 2022Meta-Learning Priors for Safe Bayesian Optimizationcited by 14
  5. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  6. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  7. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  8. 2023Thompson Sampling with Diffusion Generative Priorcited by 7
  9. 2022Meta-Learning Hypothesis Spaces for Sequential Decision-makingcited by 5
  10. 2022MARS: Meta-Learning as Score Matching in the Function Spacecited by 5
  11. 2022Lifelong Bandit Optimization: No Prior and No Regretcited by 4
  12. 2023DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learningcited by 4
11 more
  1. 2023Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networkscited by 4
  2. 2021Bayesian Active Meta-Learning for Few Pilot Demodulation and Equalizationcited by 2
  3. 2021Bayesian Active Meta-Learning for Black-Box Optimizationcited by 1
  4. 2023Model-Assisted Probabilistic Safe Adaptive Control With Meta-Bayesian Learningcited by 1
  5. 2021Transfer Bayesian Meta-learning via Weighted Free Energy Minimizationcited by 0
  6. 2021Meta-learning Amidst Heterogeneity and Ambiguitycited by 0
  7. 2023Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimationcited by 0
  8. 2025Simulation Priors for Data-Efficient Deep Learningcited by 0
  9. 2026Amortising Inference and Meta-Learning Priors in Neural Networkscited by 0
  10. 2026Regime-Conditioned Evaluation in Multi-Context Bayesian Optimizationcited by 0
  11. 2026Privacy-Aware Collaborative and Distributed Bayesian Optimizationcited by 0