# A Bayesian/Information Theoretic Model of Bias Learning

arXiv: https://arxiv.org/abs/1911.06129 · HTML: https://mlsanity.com/paper/1911.06129/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 4 of 4)

- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Pre-training helps Bayesian optimization too](https://mlsanity.com/paper/2207.03084/) (2022)
- [HyperBO+: Pre-training a universal prior for Bayesian optimization with hierarchical Gaussian processes](https://mlsanity.com/paper/2212.10538/) (2022)
- [High Efficiency Pedestrian Crossing Prediction](https://mlsanity.com/paper/2204.01862/) (2022)

## Abstract (arXiv)

In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an em objective prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machine learning problems, although we don't know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by sampling from the objective prior. Bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, and the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous.