# Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goal

arXiv: https://arxiv.org/abs/2303.01560 · HTML: https://mlsanity.com/paper/2303.01560/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 174 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Bayesian optimization for materials design](https://mlsanity.com/paper/1506.01349/) (2015)
- [Parallel Bayesian Global Optimization of Expensive Functions](https://mlsanity.com/paper/1602.05149/) (2016)
- [Active Learning for Convolutional Neural Networks: A Core-Set Approach](https://mlsanity.com/paper/1708.00489/) (2017)
- [Pool-Based Sequential Active Learning for Regression](https://mlsanity.com/paper/1805.04735/) (2018)
- [Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions](https://mlsanity.com/paper/1511.07130/) (2015)
- [Sequential Gallery for Interactive Visual Design Optimization](https://mlsanity.com/paper/2005.04107/) (2020)
- [Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization](https://mlsanity.com/paper/2011.08541/) (2020)
- [Cost-Accuracy Aware Adaptive Labeling for Active Learning](https://mlsanity.com/paper/2105.11418/) (2021)
- [Financial Applications of Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/1903.04841/) (2019)
- [An efficient application of Bayesian optimization to an industrial MDO framework for aircraft design](https://mlsanity.com/paper/2006.08434/) (2020)
- [Analytical Benchmark Problems for Multifidelity Optimization Methods](https://mlsanity.com/paper/2204.07867/) (2022)
- [Rectified Max-Value Entropy Search for Bayesian Optimization](https://mlsanity.com/paper/2202.13597/) (2022)
- [Learning cardiac activation maps from 12-lead ECG with multi-fidelity Bayesian optimization on manifolds](https://mlsanity.com/paper/2203.06222/) (2022)
- [Multi-objective and multi-fidelity Bayesian optimization of laser-plasma acceleration](https://mlsanity.com/paper/2210.03484/) (2022)
- [A novel active learning framework for classification: using weighted rank aggregation to achieve multiple query criteria](https://mlsanity.com/paper/1809.10565/) (2018)
- [Offline EEG-Based Driver Drowsiness Estimation Using Enhanced Batch-Mode Active Learning (EBMAL) for Regression](https://mlsanity.com/paper/1805.04737/) (2018)

## Built on by (most cited first, 17 of 17)

- [Efficient Evaluation of LLM Performance with Statistical Guarantees](https://mlsanity.com/paper/2601.20251/) (2026)
- [Multi-Fidelity Active Learning with GFlowNets](https://mlsanity.com/paper/2306.11715/) (2023)
- [Bayesian Active Learning with Gaussian Processes Guided by LLM Relevance Scoring for Dense Passage Retrieval](https://mlsanity.com/paper/2604.17906/) (2026)
- [Adaptive Sampling to Reduce Epistemic Uncertainty Using Prediction Interval-Generation Neural Networks](https://mlsanity.com/paper/2412.10570/) (2024)
- [Cost-Efficient Online Decision Making: A Combinatorial Multi-Armed Bandit Approach](https://mlsanity.com/paper/2308.10699/) (2023)
- [Efficient dataset construction using active learning and uncertainty-aware neural networks for plasma turbulent transport surrogate models](https://mlsanity.com/paper/2507.15976/) (2025)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [Large-scale Pretraining Improves Sample Efficiency of Active Learning based Molecule Virtual Screening](https://mlsanity.com/paper/2309.11687/) (2023)
- [Active learning for regression in engineering populations: A risk-informed approach](https://mlsanity.com/paper/2409.04328/) (2024)
- [Lecture notes on Machine Learning applications for global fits](https://mlsanity.com/paper/2604.07520/) (2026)
- [Cost-Aware Recovery-Pathway Identification and Bayesian Optimization for Autonomous Materials Discovery](https://mlsanity.com/paper/2607.23896/) (2026)
- [A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks](https://mlsanity.com/paper/2402.18729/) (2024)
- [Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation](https://mlsanity.com/paper/2510.26656/) (2025)
- [Agentic AI for Self-Driving Laboratories in Soft Matter: Taxonomy, Benchmarks,and Open Challenges](https://mlsanity.com/paper/2601.17920/) (2026)
- [Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inference](https://mlsanity.com/paper/2602.06104/) (2026)
- [How Low Can You Go? Active Learning for Sparse Model Discovery in the Ultra-Low-Data Limit](https://mlsanity.com/paper/2606.12182/) (2026)
- [Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery](https://mlsanity.com/paper/2608.04651/) (2026)

## Abstract (arXiv)

Science and Engineering applications are typically associated with expensive optimization problems to identify optimal design solutions and states of the system of interest. Bayesian optimization and active learning compute surrogate models through efficient adaptive sampling schemes to assist and accelerate this search task toward a given optimization goal. Both those methodologies are driven by specific infill/learning criteria which quantify the utility with respect to the set goal of evaluating the objective function for unknown combinations of optimization variables. While the two fields have seen an exponential growth in popularity in the past decades, their dualism and synergy have received relatively little attention to date. This paper discusses and formalizes the synergy between Bayesian optimization and active learning as symbiotic adaptive sampling methodologies driven by common principles. In particular, we demonstrate this unified perspective through the formalization of the analogy between the Bayesian infill criteria and active learning criteria as driving principles of both the goal-driven procedures. To support our original perspective, we propose a general classification of adaptive sampling techniques to highlight similarities and differences between the vast families of adaptive sampling, active learning, and Bayesian optimization. Accordingly, the synergy is demonstrated mapping the Bayesian infill criteria with the active learning criteria, and is formalized for searches informed by both a single information source and multiple levels of fidelity. In addition, we provide guidelines to apply those learning criteria investigating the performance of different Bayesian schemes for a variety of benchmark problems to highlight benefits and limitations over mathematical properties that characterize real-world applications.