# Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domains

arXiv: https://arxiv.org/abs/2106.01309 · HTML: https://mlsanity.com/paper/2106.01309/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 64 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 9 of 9)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Bayesian optimization for materials design](https://mlsanity.com/paper/1506.01349/) (2015)
- [Accelerating high-throughput virtual screening through molecular pool-based active learning](https://mlsanity.com/paper/2012.07127/) (2020)
- [Batched High-dimensional Bayesian Optimization via Structural Kernel Learning](https://mlsanity.com/paper/1703.01973/) (2017)
- [Olympus: a benchmarking framework for noisy optimization and experiment planning](https://mlsanity.com/paper/2010.04153/) (2020)
- [Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution](https://mlsanity.com/paper/1712.08048/) (2017)

## Built on by (most cited first, 30 of 48)

- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Bayesian Optimization of Catalysis With In-Context Learning](https://mlsanity.com/paper/2304.05341/) (2023)
- [Autonomous optimization of nonaqueous battery electrolytes via robotic experimentation and machine learning](https://mlsanity.com/paper/2111.14786/) (2021)
- [Randomized Gaussian Process Upper Confidence Bound with Tighter Bayesian Regret Bounds](https://mlsanity.com/paper/2302.01511/) (2023)
- [Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)](https://mlsanity.com/paper/2208.13771/) (2022)
- [Benchmarking Active Learning Strategies for Materials Optimization and Discovery](https://mlsanity.com/paper/2204.05838/) (2022)
- [Towards out of distribution generalization for problems in mechanics](https://mlsanity.com/paper/2206.14917/) (2022)
- [EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions](https://mlsanity.com/paper/2106.04618/) (2021)
- [Machine Learning with Knowledge Constraints for Process Optimization of Open-Air Perovskite Solar Cell Manufacturing](https://mlsanity.com/paper/2110.01387/) (2021)
- [Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design](https://mlsanity.com/paper/2207.04994/) (2022)
- [Towards Futuristic Autonomous Experimentation--A Surprise-Reacting Sequential Experiment Policy](https://mlsanity.com/paper/2112.00600/) (2021)
- [Regret Analysis for Randomized Gaussian Process Upper Confidence Bound](https://mlsanity.com/paper/2409.00979/) (2024)
- [Bayesian Optimisation Against Climate Change: Applications and Benchmarks](https://mlsanity.com/paper/2306.04343/) (2023)
- [Accelerating material discovery with a threshold-driven hybrid acquisition policy-based Bayesian optimization](https://mlsanity.com/paper/2311.09591/) (2023)
- [Training-Free Active Learning Framework in Materials Science with Large Language Models](https://mlsanity.com/paper/2511.19730/) (2025)
- [Engineering Regression Without Real-Data Training: Domain Adaptation for Tabular Foundation Models Using Multi-Dataset Embeddings](https://mlsanity.com/paper/2603.04692/) (2026)
- [Adaptive AI decision interface for autonomous electronic material discovery](https://mlsanity.com/paper/2504.13344/) (2025)
- [Position: Intelligent Science Laboratory Requires the Integration of Cognitive and Embodied AI](https://mlsanity.com/paper/2506.19613/) (2025)
- [Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores](https://mlsanity.com/paper/2512.01080/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [In-Context Black-Box Optimization with Unreliable Feedback](https://mlsanity.com/paper/2605.06187/) (2026)
- [DASH: Decoupled Adaptive Surrogate - Acquisition Harness for Automated Bayesian Optimization](https://mlsanity.com/paper/2608.00641/) (2026)
- [Long-run Behaviour of Multi-fidelity Bayesian Optimisation](https://mlsanity.com/paper/2312.12633/) (2023)
- [Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions](https://mlsanity.com/paper/2504.08874/) (2025)
- [Interpretable SHAP-bounded Bayesian Optimization for Underwater Acoustic Metamaterial Coating Design](https://mlsanity.com/paper/2505.06519/) (2025)
- [Machine learning for in-situ composition mapping in a self-driving magnetron sputtering system](https://mlsanity.com/paper/2506.05999/) (2025)
- [A Guide to Bayesian Optimization in Bioprocess Engineering](https://mlsanity.com/paper/2508.10642/) (2025)
- [Foundation-Model Surrogates Enable Data-Efficient Active Learning for Materials Discovery](https://mlsanity.com/paper/2603.12567/) (2026)
- [BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks](https://mlsanity.com/paper/2605.17000/) (2026)
- [Opportunities for Machine Learning to Accelerate Halide Perovskite Commercialization and Scale-Up](https://mlsanity.com/paper/2110.03923/) (2021)

## Abstract (arXiv)

In the field of machine learning (ML) for materials optimization, active learning algorithms, such as Bayesian Optimization (BO), have been leveraged for guiding autonomous and high-throughput experimentation systems. However, very few studies have evaluated the efficiency of BO as a general optimization algorithm across a broad range of experimental materials science domains. In this work, we evaluate the performance of BO algorithms with a collection of surrogate model and acquisition function pairs across five diverse experimental materials systems, namely carbon nanotube polymer blends, silver nanoparticles, lead-halide perovskites, as well as additively manufactured polymer structures and shapes. By defining acceleration and enhancement metrics for general materials optimization objectives, we find that for surrogate model selection, Gaussian Process (GP) with anisotropic kernels (automatic relevance detection, ARD) and Random Forests (RF) have comparable performance and both outperform the commonly used GP without ARD. We discuss the implicit distributional assumptions of RF and GP, and the benefits of using GP with anisotropic kernels in detail. We provide practical insights for experimentalists on surrogate model selection of BO during materials optimization campaigns.