# Bayesian Optimization with Rich Auxiliary Information via LLMs

arXiv: https://arxiv.org/abs/2609.19437 · HTML: https://mlsanity.com/paper/2609.19437/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 70 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [Freeze-Thaw Bayesian Optimization](https://mlsanity.com/paper/1406.3896/) (2014)
- [ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations](https://mlsanity.com/paper/1908.01425/) (2019)
- [Using Large Language Models for Hyperparameter Optimization](https://mlsanity.com/paper/2312.04528/) (2023)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?](https://mlsanity.com/paper/2402.05015/) (2024)
- [ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models](https://mlsanity.com/paper/2406.18770/) (2024)
- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [Bayesian Optimization with Binary Auxiliary Information](https://mlsanity.com/paper/1906.07277/) (2019)
- [LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization](https://mlsanity.com/paper/2505.14756/) (2025)
- [LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?](https://mlsanity.com/paper/2509.21403/) (2025)
- [Correcting boundary over-exploration deficiencies in Bayesian optimization with virtual derivative sign observations](https://mlsanity.com/paper/1704.00963/) (2017)
- [Large language models as uncertainty-calibrated optimizers for experimental discovery](https://mlsanity.com/paper/2504.06265/) (2025)
- [Adaptive Acquisition Selection for Bayesian Optimization with Large Language Models](https://mlsanity.com/paper/2602.07904/) (2026)
- [Offline Contextual Bayesian Optimization for Nuclear Fusion](https://mlsanity.com/paper/2001.01793/) (2020)
- [Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery](https://mlsanity.com/paper/2605.17976/) (2026)
- [Multi-Timescale Dynamics Model Bayesian Optimization for Plasma Stabilization in Tokamaks](https://mlsanity.com/paper/2506.10287/) (2025)
- [LILO: Bayesian Optimization with Natural Language Feedback](https://mlsanity.com/paper/2510.17671/) (2025)
- [Efficient Transfer Bayesian Optimization with Auxiliary Information](https://mlsanity.com/paper/1909.07670/) (2019)
- [Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization](https://mlsanity.com/paper/2511.02570/) (2025)

## Abstract (arXiv)

Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, yet it typically reduces each expensive experiment to an input and its objective value, discarding much of the information the experiment produces. Such auxiliary information can include training curves in hyperparameter optimization, expert notes and images in scientific experimentation, or known facts about the specific optimization problem. Leveraging the ability of large language models (LLMs) to process diverse and unstructured information, we introduce two methods: AuxBO-Evolve, which uses auxiliary information to evolve beliefs over the location of the optimum, and AuxBO-Vicinity, which uses it to locally guide the acquisition function. Across synthetic tasks, hyperparameter optimization benchmarks, and a nuclear fusion task with unstructured scientist-written logs, our methods consistently improve optimization performance and outperform standard BO and existing LLM-based approaches. We further find that LLMs provide more effective probabilistic guidance when modeling likely maximizer locations rather than objective values pointwise. Overall, our results demonstrate that exploiting rich information beyond standard (x, y) interactions can substantially improve BO performance.