# Learning to Learn Kernels with Variational Random Features

arXiv: https://arxiv.org/abs/2006.06707 · HTML: https://mlsanity.com/paper/2006.06707/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 18 of 18)

- [Kernel Continual Learning](https://mlsanity.com/paper/2107.05757/) (2021)
- [Realistic Evaluation of Transductive Few-Shot Learning](https://mlsanity.com/paper/2204.11181/) (2022)
- [Hierarchical Variational Memory for Few-shot Learning Across Domains](https://mlsanity.com/paper/2112.08181/) (2021)
- [Learning to Rectify for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2111.04239/) (2021)
- [SuperDisco: Super-Class Discovery Improves Visual Recognition for the Long-Tail](https://mlsanity.com/paper/2304.00101/) (2023)
- [Transductive Few-Shot Learning: Clustering is All You Need?](https://mlsanity.com/paper/2106.09516/) (2021)
- [Towards Enabling Meta-Learning from Target Models](https://mlsanity.com/paper/2104.03736/) (2021)
- [MetaKernel: Learning Variational Random Features with Limited Labels](https://mlsanity.com/paper/2105.03781/) (2021)
- [On The Relative Error of Random Fourier Features for Preserving Kernel Distance](https://mlsanity.com/paper/2210.00244/) (2022)
- [EMO: Episodic Memory Optimization for Few-Shot Meta-Learning](https://mlsanity.com/paper/2306.05189/) (2023)
- [Random feature approximation for general spectral methods](https://mlsanity.com/paper/2506.16283/) (2025)
- [Scalable Random Wavelet Features: Efficient Non-Stationary Kernel Approximation with Convergence Guarantees](https://mlsanity.com/paper/2602.00987/) (2026)
- [Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators](https://mlsanity.com/paper/2603.00971/) (2026)
- [Kernel-Based Models for Influence Maximization on Graphs based on Gaussian Process Variance Minimization](https://mlsanity.com/paper/2103.01575/) (2021)
- [Random feature approximation for general spectral methods](https://mlsanity.com/paper/2308.15434/) (2023)
- [Stein Random Feature Regression](https://mlsanity.com/paper/2406.00438/) (2024)
- [Kernel Learning for Sample Constrained Black-Box Optimization](https://mlsanity.com/paper/2507.20533/) (2025)
- [Deep kernel hedging](https://mlsanity.com/paper/2609.34474/) (2026)

## Abstract (arXiv)

In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational random features (MetaVRF) to learn adaptive kernels for the base-learner, which is developed in a latent variable model by treating the random feature basis as the latent variable. We formulate the optimization of MetaVRF as a variational inference problem by deriving an evidence lower bound under the meta-learning framework. To incorporate shared knowledge from related tasks, we propose a context inference of the posterior, which is established by an LSTM architecture. The LSTM-based inference network can effectively integrate the context information of previous tasks with task-specific information, generating informative and adaptive features. The learned MetaVRF can produce kernels of high representational power with a relatively low spectral sampling rate and also enables fast adaptation to new tasks. Experimental results on a variety of few-shot regression and classification tasks demonstrate that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.