# Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

arXiv: https://arxiv.org/abs/2607.29225 · HTML: https://mlsanity.com/paper/2607.29225/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 70 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [A Study of Bayesian Neural Network Surrogates for Bayesian Optimization](https://mlsanity.com/paper/2305.20028/) (2023)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)](https://mlsanity.com/paper/2208.13771/) (2022)
- [Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations](https://mlsanity.com/paper/2403.08151/) (2024)
- [Memory-Based Dual Gaussian Processes for Sequential Learning](https://mlsanity.com/paper/2306.03566/) (2023)
- [Calibration and generalizability of probabilistic models on low-data chemical datasets with DIONYSUS](https://mlsanity.com/paper/2212.01574/) (2022)
- [Mondrian Forests for Large-Scale Regression when Uncertainty Matters](https://mlsanity.com/paper/1506.03805/) (2015)
- [Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes](https://mlsanity.com/paper/2412.20375/) (2024)
- [BeBOP -- Combining Reactive Planning and Bayesian Optimization to Solve Robotic Manipulation Tasks](https://mlsanity.com/paper/2310.00971/) (2023)
- [Decreasing the Computing Time of Bayesian Optimization using Generalizable Memory Pruning](https://mlsanity.com/paper/2309.04510/) (2023)

## Built on by (most cited first, 1 of 1)

- [Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery](https://mlsanity.com/paper/2608.04651/) (2026)

## Abstract (arXiv)

Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance. Here we present a systematic, compute-aware study of BO that evaluates surrogate models along two axes: optimization quality and computational frugality. Across eight benchmark functions and nine real-world datasets spanning materials science, mechanics, robotics, chemistry, and machine learning, we benchmark four surrogate models: Gaussian Processes, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces. We show that Gaussian Process-based BO consistently incurs the highest time and memory overhead without delivering superior optimization or sample efficiency. In contrast, scalable alternatives achieve equal or better performance at a fraction of the computational cost. Motivated by these findings, we introduce a surrogate-recommendation framework that predicts the most suitable BO surrogate from inexpensive dataset characteristics. Together, these results establish FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provide practical guidance for surrogate selection under limited computational and experimental budgets.