# Uncertainty quantification and exploration-exploitation trade-off in humans

arXiv: https://arxiv.org/abs/2102.07647 · HTML: https://mlsanity.com/paper/2102.07647/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 76 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 20 of 20)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [An Information-Theoretic Analysis of Thompson Sampling](https://mlsanity.com/paper/1403.5341/) (2014)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizations](https://mlsanity.com/paper/1805.12168/) (2018)
- [Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation](https://mlsanity.com/paper/1911.12809/) (2019)
- [Randomised Gaussian Process Upper Confidence Bound for Bayesian Optimisation](https://mlsanity.com/paper/2006.04296/) (2020)
- [Mean-Variance Analysis in Bayesian Optimization under Uncertainty](https://mlsanity.com/paper/2009.08166/) (2020)
- [Optimization as Estimation with Gaussian Processes in Bandit Settings](https://mlsanity.com/paper/1510.06423/) (2015)
- [Global optimization via inverse distance weighting and radial basis functions](https://mlsanity.com/paper/1906.06498/) (2019)
- [Uncertainty quantification using martingales for misspecified Gaussian processes](https://mlsanity.com/paper/2006.07368/) (2020)
- [The Human Kernel](https://mlsanity.com/paper/1510.07389/) (2015)
- [Efficient sampling for Gaussian linear regression with arbitrary priors](https://mlsanity.com/paper/1806.05738/) (2018)
- [ε-shotgun: ε-greedy Batch Bayesian Optimisation](https://mlsanity.com/paper/2002.01873/) (2020)
- [Modelling Human Active Search in Optimizing Black-box Functions](https://mlsanity.com/paper/2003.04275/) (2020)
- [Fortified Test Functions for Global Optimization and the Power of Multiple Runs](https://mlsanity.com/paper/1912.10575/) (2019)
- [Exact Power of the Rank-Sum Test for a Continuous Variable](https://mlsanity.com/paper/1901.04597/) (2019)

## Built on by (most cited first, 1 of 1)

- [Mastering the exploration-exploitation trade-off in Bayesian Optimization](https://mlsanity.com/paper/2305.08624/) (2023)

## Abstract (arXiv)

The main objective of this paper is to outline a theoretical framework to analyse how humans' decision-making strategies under uncertainty manage the trade-off between information gathering (exploration) and reward seeking (exploitation). A key observation, motivating this line of research, is the awareness that human learners are amazingly fast and effective at adapting to unfamiliar environments and incorporating upcoming knowledge: this is an intriguing behaviour for cognitive sciences as well as an important challenge for Machine Learning. The target problem considered is active learning in a black-box optimization task and more specifically how the exploration/exploitation dilemma can be modelled within Gaussian Process based Bayesian Optimization framework, which is in turn based on uncertainty quantification. The main contribution is to analyse humans' decisions with respect to Pareto rationality where the two objectives are improvement expected and uncertainty quantification. According to this Pareto rationality model, if a decision set contains a Pareto efficient (dominant) strategy, a rational decision maker should always select the dominant strategy over its dominated alternatives. The distance from the Pareto frontier determines whether a choice is (Pareto) rational (i.e., lays on the frontier) or is associated to "exasperate" exploration. However, since the uncertainty is one of the two objectives defining the Pareto frontier, we have investigated three different uncertainty quantification measures and selected the one resulting more compliant with the Pareto rationality model proposed. The key result is an analytical framework to characterize how deviations from "rationality" depend on uncertainty quantifications and the evolution of the reward seeking process.