# Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task

arXiv: https://arxiv.org/abs/2104.09237 · HTML: https://mlsanity.com/paper/2104.09237/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Inverse Optimization with Noisy Data](https://mlsanity.com/paper/1507.03266/) (2015)
- [Data-driven Inverse Optimization with Imperfect Information](https://mlsanity.com/paper/1512.05489/) (2015)
- [Inverse Optimization: Closed-form Solutions, Geometry and Goodness of fit](https://mlsanity.com/paper/1511.04650/) (2015)
- [The Human Kernel](https://mlsanity.com/paper/1510.07389/) (2015)
- [Inverse Optimization of Convex Risk Functions](https://mlsanity.com/paper/1607.07099/) (2016)
- [Quantile Inverse Optimization: Improving Stability in Inverse Linear Programming](https://mlsanity.com/paper/1908.02376/) (2019)
- [Modelling Human Active Search in Optimizing Black-box Functions](https://mlsanity.com/paper/2003.04275/) (2020)

## Built on by (most cited first, 1 of 1)

- [Systematic Evaluation of Initial States and Exploration-Exploitation Strategies in PID Auto-Tuning: A Framework-Driven Approach Applied on Mobile Robots](https://mlsanity.com/paper/2505.03159/) (2025)

## Abstract (arXiv)

This paper introduces a probabilistic framework to estimate parameters of an acquisition function given observed human behavior that can be modeled as a collection of sample paths from a Bayesian optimization procedure. The methodology involves defining a likelihood on observed human behavior from an optimization task, where the likelihood is parameterized by a Bayesian optimization subroutine governed by an unknown acquisition function. This structure enables us to make inference on a subject's acquisition function while allowing their behavior to deviate around the solution to the Bayesian optimization subroutine. To test our methods, we designed a sequential optimization task which forced subjects to balance exploration and exploitation in search of an invisible target location. Applying our proposed methods to the resulting data, we find that many subjects tend to exhibit exploration preferences beyond that of standard acquisition functions to capture. Guided by the model discrepancies, we augment the candidate acquisition functions to yield a superior fit to the human behavior in this task.