# Meta-Learning Priors for Efficient Online Bayesian Regression

arXiv: https://arxiv.org/abs/1807.08912 · HTML: https://mlsanity.com/paper/1807.08912/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2018.

## Built on by (most cited first, 30 of 48)

- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems](https://mlsanity.com/paper/2103.04490/) (2021)
- [A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs](https://mlsanity.com/paper/2110.13361/) (2021)
- [A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases](https://mlsanity.com/paper/2209.11208/) (2022)
- [Deep Generative Symbolic Regression](https://mlsanity.com/paper/2401.00282/) (2023)
- [Meta-Learning Reliable Priors in the Function Space](https://mlsanity.com/paper/2106.03195/) (2021)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning](https://mlsanity.com/paper/2209.11820/) (2022)
- [Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms](https://mlsanity.com/paper/2103.03697/) (2021)
- [Control-oriented meta-learning](https://mlsanity.com/paper/2204.06716/) (2022)
- [Meta-Learning Priors for Safe Bayesian Optimization](https://mlsanity.com/paper/2210.00762/) (2022)
- [Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control](https://mlsanity.com/paper/2210.12583/) (2022)
- [Non-Gaussian Gaussian Processes for Few-Shot Regression](https://mlsanity.com/paper/2110.13561/) (2021)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Dual Online Stein Variational Inference for Control and Dynamics](https://mlsanity.com/paper/2103.12890/) (2021)
- [Bayesian Embeddings for Few-Shot Open World Recognition](https://mlsanity.com/paper/2107.13682/) (2021)
- [Graph Reinforcement Learning for Network Control via Bi-Level Optimization](https://mlsanity.com/paper/2305.09129/) (2023)
- [Learning to Continually Learn with the Bayesian Principle](https://mlsanity.com/paper/2405.18758/) (2024)
- [Adaptive Robust Model Predictive Control with Matched and Unmatched Uncertainty](https://mlsanity.com/paper/2104.08261/) (2021)
- [Adaptive Control of SE(3) Hamiltonian Dynamics with Learned Disturbance Features](https://mlsanity.com/paper/2109.09974/) (2021)
- [Meta Learning of Interface Conditions for Multi-Domain Physics-Informed Neural Networks](https://mlsanity.com/paper/2210.12669/) (2022)
- [Meta-Learning with Adjoint Methods](https://mlsanity.com/paper/2110.08432/) (2021)
- [End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimization](https://mlsanity.com/paper/2111.00639/) (2021)
- [Tractable Uncertainty-Aware Meta-Learning](https://mlsanity.com/paper/2210.01881/) (2022)
- [Lifelong Bandit Optimization: No Prior and No Regret](https://mlsanity.com/paper/2210.15513/) (2022)
- [A New First-Order Meta-Learning Algorithm with Convergence Guarantees](https://mlsanity.com/paper/2409.03682/) (2024)
- [Bayesian Meta-Learning Through Variational Gaussian Processes](https://mlsanity.com/paper/2110.11044/) (2021)
- [First, Learn What You Don't Know: Active Information Gathering for Driving at the Limits of Handling](https://mlsanity.com/paper/2411.00107/) (2024)
- [Meta-Learning Online Dynamics Model Adaptation in Off-Road Autonomous Driving](https://mlsanity.com/paper/2504.16923/) (2025)
- [Meta-free few-shot learning via representation learning with weight averaging](https://mlsanity.com/paper/2204.12466/) (2022)

## Abstract (arXiv)

Gaussian Process (GP) regression has seen widespread use in robotics due to its generality, simplicity of use, and the utility of Bayesian predictions. The predominant implementation of GP regression is a nonparameteric kernel-based approach, as it enables fitting of arbitrary nonlinear functions. However, this approach suffers from two main drawbacks: (1) it is computationally inefficient, as computation scales poorly with the number of samples; and (2) it can be data inefficient, as encoding prior knowledge that can aid the model through the choice of kernel and associated hyperparameters is often challenging and unintuitive. In this work, we propose ALPaCA, an algorithm for efficient Bayesian regression which addresses these issues. ALPaCA uses a dataset of sample functions to learn a domain-specific, finite-dimensional feature encoding, as well as a prior over the associated weights, such that Bayesian linear regression in this feature space yields accurate online predictions of the posterior predictive density. These features are neural networks, which are trained via a meta-learning (or "learning-to-learn") approach. ALPaCA extracts all prior information directly from the dataset, rather than restricting prior information to the choice of kernel hyperparameters. Furthermore, by operating in the weight space, it substantially reduces sample complexity. We investigate the performance of ALPaCA on two simple regression problems, two simulated robotic systems, and on a lane-change driving task performed by humans. We find our approach outperforms kernel-based GP regression, as well as state of the art meta-learning approaches, thereby providing a promising plug-in tool for many regression tasks in robotics where scalability and data-efficiency are important.