# Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)

arXiv: https://arxiv.org/abs/2208.13771 · HTML: https://mlsanity.com/paper/2208.13771/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 75 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 21 of 21)

- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [Local Latent Space Bayesian Optimization over Structured Inputs](https://mlsanity.com/paper/2201.11872/) (2022)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [Batched High-dimensional Bayesian Optimization via Structural Kernel Learning](https://mlsanity.com/paper/1703.01973/) (2017)
- [Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domains](https://mlsanity.com/paper/2106.01309/) (2021)
- [High Dimensional Bayesian Optimization Using Dropout](https://mlsanity.com/paper/1802.05400/) (2018)
- [A Tutorial on Sparse Gaussian Processes and Variational Inference](https://mlsanity.com/paper/2012.13962/) (2020)
- [Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge](https://mlsanity.com/paper/2003.12127/) (2020)
- [TREGO: a Trust-Region Framework for Efficient Global Optimization](https://mlsanity.com/paper/2101.06808/) (2021)
- [Time Efficiency in Optimization with a Bayesian-Evolutionary Algorithm](https://mlsanity.com/paper/2005.04166/) (2020)
- [Machine Learning with Knowledge Constraints for Process Optimization of Open-Air Perovskite Solar Cell Manufacturing](https://mlsanity.com/paper/2110.01387/) (2021)
- [Machine-learned metrics for predicting the likelihood of success in materials discovery](https://mlsanity.com/paper/1911.11201/) (2019)
- [FLASH: Fast Bayesian Optimization for Data Analytic Pipelines](https://mlsanity.com/paper/1602.06468/) (2016)
- [A Machine Learning and Computer Vision Approach to Rapidly Optimize Multiscale Droplet Generation](https://mlsanity.com/paper/2105.13553/) (2021)
- [Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure](https://mlsanity.com/paper/1403.6573/) (2014)
- [Fast Hyperparameter Tuning using Bayesian Optimization with Directional Derivatives](https://mlsanity.com/paper/1902.02416/) (2019)
- [Needle in a Haystack: Label-Efficient Evaluation under Extreme Class Imbalance](https://mlsanity.com/paper/2006.06963/) (2020)

## Built on by (most cited first, 14 of 14)

- [Deep Reinforcement Learning for Inverse Inorganic Materials Design](https://mlsanity.com/paper/2210.11931/) (2022)
- [Language-Based Bayesian Optimization Research Assistant (BORA)](https://mlsanity.com/paper/2501.16224/) (2025)
- [Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal](https://mlsanity.com/paper/2410.10384/) (2024)
- [HypBO: Accelerating Black-Box Scientific Experiments Using Experts' Hypotheses](https://mlsanity.com/paper/2308.11787/) (2023)
- [NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Information](https://mlsanity.com/paper/2510.05516/) (2025)
- [Domain Knowledge Injection in Bayesian Search for New Materials](https://mlsanity.com/paper/2311.15162/) (2023)
- [BEACON: A Bayesian Optimization Inspired Strategy for Efficient Novelty Search](https://mlsanity.com/paper/2406.03616/) (2024)
- [Decreasing the Computing Time of Bayesian Optimization using Generalizable Memory Pruning](https://mlsanity.com/paper/2309.04510/) (2023)
- [Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputs](https://mlsanity.com/paper/2605.10385/) (2026)
- [Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery](https://mlsanity.com/paper/2607.29225/) (2026)
- [SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality](https://mlsanity.com/paper/2406.12661/) (2024)
- [Multi-Variable Batch Bayesian Optimization in Materials Research: Synthetic Data Analysis of Noise Sensitivity and Problem Landscape Effects](https://mlsanity.com/paper/2504.03943/) (2025)
- [tidyHEBO: Improving the Robustness of Heteroskedastic Evolutionary Bayesian Optimization](https://mlsanity.com/paper/2607.10669/) (2026)
- [Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery](https://mlsanity.com/paper/2608.04651/) (2026)

## Abstract (arXiv)

Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. For example, only 0.82% out of 146k total materials in the open-access Materials Project database have a negative Poisson's ratio. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from O(n3) to O(φ3) for φ forward experiments per activation, which trends to a constant O(1) over several activations. Additionally, ZoMBI implements two custom adaptive acquisition functions to further guide the sampling of new experiments toward the global optimum. We validate the algorithm's optimization performance on three real-world datasets exhibiting Needle-in-a-Haystack and further stress-test the algorithm's performance on an additional 174 analytical datasets. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods MiP-EGO, TuRBO, and HEBO.