# Dynamic Exploration-Exploitation Trade-Off in Active Learning Regression with Bayesian Hierarchical Modeling

arXiv: https://arxiv.org/abs/2304.07665 · HTML: https://mlsanity.com/paper/2304.07665/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 80 references that are in the map.
- First posted on arXiv 16 Apr 2023.

## Builds on (most cited first, 6 of 6)

- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [Algorithms for multi-armed bandit problems](https://mlsanity.com/paper/1402.6028/) (2014)
- [Active Learning for Regression Using Greedy Sampling](https://mlsanity.com/paper/1808.04245/) (2018)
- [A hierarchical expected improvement method for Bayesian optimization](https://mlsanity.com/paper/1911.07285/) (2019)
- [A Variance Maximization Criterion for Active Learning](https://mlsanity.com/paper/1706.07642/) (2017)
- [Auxiliary Likelihood-Based Approximate Bayesian Computation in State Space Models](https://mlsanity.com/paper/1604.07949/) (2016)

## Built on by (most cited first, 2 of 2)

- [Active Learning Using Aggregated Acquisition Functions: Accuracy and Sustainability Analysis](https://mlsanity.com/paper/2602.07440/) (2026)
- [Mutual Information Surprise: Rethinking Unexpectedness in Autonomous Systems](https://mlsanity.com/paper/2508.17403/) (2025)

## Abstract (arXiv)

Active learning provides a framework to adaptively query the most informative experiments towards learning an unknown black-box function. Various approaches of active learning have been proposed in the literature, however, they either focus on exploration or exploitation in the design space. Methods that do consider exploration-exploitation simultaneously employ fixed or ad-hoc measures to control the trade-off that may not be optimal. In this paper, we develop a Bayesian hierarchical approach, referred as BHEEM, to dynamically balance the exploration-exploitation trade-off as more data points are queried. To sample from the posterior distribution of the trade-off parameter, We subsequently formulate an approximate Bayesian computation approach based on the linear dependence of queried data in the feature space. Simulated and real-world examples show the proposed approach achieves at least 21% and 11% average improvement when compared to pure exploration and exploitation strategies respectively. More importantly, we note that by optimally balancing the trade-off between exploration and exploitation, BHEEM performs better or at least as well as either pure exploration or pure exploitation.