# Non-Gaussian Gaussian Processes for Few-Shot Regression

arXiv: https://arxiv.org/abs/2110.13561 · HTML: https://mlsanity.com/paper/2110.13561/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Meta-Learning without Memorization](https://mlsanity.com/paper/1912.03820/) (2019)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Meta-Learning Priors for Efficient Online Bayesian Regression](https://mlsanity.com/paper/1807.08912/) (2018)
- [MetaFun: Meta-Learning with Iterative Functional Updates](https://mlsanity.com/paper/1912.02738/) (2019)
- [Bayesian Few-Shot Classification with One-vs-Each P'olya-Gamma Augmented Gaussian Processes](https://mlsanity.com/paper/2007.10417/) (2020)
- [Transforming Gaussian Processes With Normalizing Flows](https://mlsanity.com/paper/2011.01596/) (2020)
- [Gaussian Process Conditional Density Estimation](https://mlsanity.com/paper/1810.12750/) (2018)
- [Meta-Learning Mean Functions for Gaussian Processes](https://mlsanity.com/paper/1901.08098/) (2019)
- [Deep Structured Mixtures of Gaussian Processes](https://mlsanity.com/paper/1910.04536/) (2019)
- [Kernel Dependence Regularizers and Gaussian Processes with Applications to Algorithmic Fairness](https://mlsanity.com/paper/1911.04322/) (2019)
- [Adaptive Deep Kernel Learning](https://mlsanity.com/paper/1905.12131/) (2019)
- [Information Theoretic Meta Learning with Gaussian Processes](https://mlsanity.com/paper/2009.03228/) (2020)
- [Gaussian Process Regression for Probabilistic Short-term Solar Output Forecast](https://mlsanity.com/paper/2002.10878/) (2020)

## Built on by (most cited first, 12 of 12)

- [HyperShot: Few-Shot Learning by Kernel HyperNetworks](https://mlsanity.com/paper/2203.11378/) (2022)
- [Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation](https://mlsanity.com/paper/2408.16228/) (2024)
- [HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks](https://mlsanity.com/paper/2205.15745/) (2022)
- [Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification](https://mlsanity.com/paper/2206.09843/) (2022)
- [Sample-based Uncertainty Quantification with a Single Deterministic Neural Network](https://mlsanity.com/paper/2209.08418/) (2022)
- [Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation](https://mlsanity.com/paper/2306.13554/) (2023)
- [TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression](https://mlsanity.com/paper/2206.04140/) (2022)
- [Hypernetwork approach to Bayesian MAML](https://mlsanity.com/paper/2210.02796/) (2022)
- [Target-Free Compound Activity Prediction via Few-Shot Learning](https://mlsanity.com/paper/2311.16328/) (2023)
- [Marginalization Consistent Probabilistic Forecasting of Irregular Time Series via Mixture of Separable flows](https://mlsanity.com/paper/2406.07246/) (2024)
- [Composite Gaussian Processes Flows for Learning Discontinuous Multimodal Policies](https://mlsanity.com/paper/2502.01913/) (2025)
- [Reducing Estimation Uncertainty Using Normalizing Flows and Stratification](https://mlsanity.com/paper/2602.10706/) (2026)

## Abstract (arXiv)

Gaussian Processes (GPs) have been widely used in machine learning to model distributions over functions, with applications including multi-modal regression, time-series prediction, and few-shot learning. GPs are particularly useful in the last application since they rely on Normal distributions and enable closed-form computation of the posterior probability function. Unfortunately, because the resulting posterior is not flexible enough to capture complex distributions, GPs assume high similarity between subsequent tasks - a requirement rarely met in real-world conditions. In this work, we address this limitation by leveraging the flexibility of Normalizing Flows to modulate the posterior predictive distribution of the GP. This makes the GP posterior locally non-Gaussian, therefore we name our method Non-Gaussian Gaussian Processes (NGGPs). More precisely, we propose an invertible ODE-based mapping that operates on each component of the random variable vectors and shares the parameters across all of them. We empirically tested the flexibility of NGGPs on various few-shot learning regression datasets, showing that the mapping can incorporate context embedding information to model different noise levels for periodic functions. As a result, our method shares the structure of the problem between subsequent tasks, but the contextualization allows for adaptation to dissimilarities. NGGPs outperform the competing state-of-the-art approaches on a diversified set of benchmarks and applications.