# Language-Based Bayesian Optimization Research Assistant (BORA)

arXiv: https://arxiv.org/abs/2501.16224 · HTML: https://mlsanity.com/paper/2501.16224/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Builds on (most cited first, 30 of 34)

- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization](https://mlsanity.com/paper/1904.02642/) (2019)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models](https://mlsanity.com/paper/2406.18770/) (2024)
- [Cost-aware Bayesian Optimization](https://mlsanity.com/paper/2003.10870/) (2020)
- [Towards Optimizing with Large Language Models](https://mlsanity.com/paper/2310.05204/) (2023)
- [Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge](https://mlsanity.com/paper/2003.12127/) (2020)
- [In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics](https://mlsanity.com/paper/2410.16309/) (2024)
- [Looping in the Human Collaborative and Explainable Bayesian Optimization](https://mlsanity.com/paper/2310.17273/) (2023)
- [LICO: Large Language Models for In-Context Molecular Optimization](https://mlsanity.com/paper/2406.18851/) (2024)
- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [Incorporating Expert Prior in Bayesian Optimisation via Space Warping](https://mlsanity.com/paper/2003.12250/) (2020)
- [A General Framework for User-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.14645/) (2023)
- [Warm Starting Bayesian Optimization](https://mlsanity.com/paper/1608.03585/) (2016)
- [Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study](https://mlsanity.com/paper/2406.10675/) (2024)
- [Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models](https://mlsanity.com/paper/2404.06290/) (2024)
- [Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)](https://mlsanity.com/paper/2208.13771/) (2022)
- [Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling](https://mlsanity.com/paper/2002.11256/) (2020)
- [In-Context Explainers: Harnessing LLMs for Explaining Black Box Models](https://mlsanity.com/paper/2310.05797/) (2023)
- [Bayesian Optimization Augmented with Actively Elicited Expert Knowledge](https://mlsanity.com/paper/2208.08742/) (2022)
- [HypBO: Accelerating Black-Box Scientific Experiments Using Experts' Hypotheses](https://mlsanity.com/paper/2308.11787/) (2023)
- [The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures](https://mlsanity.com/paper/2207.12761/) (2022)
- [Accelerating material discovery with a threshold-driven hybrid acquisition policy-based Bayesian optimization](https://mlsanity.com/paper/2311.09591/) (2023)
- [Domain Knowledge Injection in Bayesian Search for New Materials](https://mlsanity.com/paper/2311.15162/) (2023)

## Built on by (most cited first, 15 of 15)

- [Large language models as uncertainty-calibrated optimizers for experimental discovery](https://mlsanity.com/paper/2504.06265/) (2025)
- [Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMs](https://mlsanity.com/paper/2505.12833/) (2025)
- [Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch](https://mlsanity.com/paper/2608.00316/) (2026)
- [Cooperative Design Optimization through Natural Language Interaction](https://mlsanity.com/paper/2508.16077/) (2025)
- [TextBO: Bayesian Optimization in Language Space for Eval-Efficient Self-Improving AI](https://mlsanity.com/paper/2511.12063/) (2025)
- [Training-Free Active Learning Framework in Materials Science with Large Language Models](https://mlsanity.com/paper/2511.19730/) (2025)
- [Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial](https://mlsanity.com/paper/2604.01328/) (2026)
- [Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH](https://mlsanity.com/paper/2601.12355/) (2026)
- [DASH: Decoupled Adaptive Surrogate - Acquisition Harness for Automated Bayesian Optimization](https://mlsanity.com/paper/2608.00641/) (2026)
- [CARE: Context-Aware Ranking Evolution with Executable Scoring Programs for Budgeted Reaction Optimization](https://mlsanity.com/paper/2606.14581/) (2026)
- [CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization](https://mlsanity.com/paper/2602.09851/) (2026)
- [Zero-Shot Active Feature Acquisition via LLM-Elicitation](https://mlsanity.com/paper/2606.18933/) (2026)
- [LLMs as Acquisition Policies for Finite-Pool Materials Optimization: A Controlled Study](https://mlsanity.com/paper/2608.19790/) (2026)
- [Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search](https://mlsanity.com/paper/2609.34281/) (2026)
- [Agentic-MatriBO: Capability-Compiled Agentic Bayesian Optimization for Materials and Chemical Design](https://mlsanity.com/paper/2610.17832/) (2026)

## Abstract (arXiv)

Many important scientific problems involve multivariate optimization coupled with slow and laborious experimental measurements. These complex, high-dimensional searches can be defined by non-convex optimization landscapes that resemble needle-in-a-haystack surfaces, leading to entrapment in local minima. Contextualizing optimizers with human domain knowledge is a powerful approach to guide searches to localized fruitful regions. However, this approach is susceptible to human confirmation bias and it is also challenging for domain experts to keep track of the rapidly expanding scientific literature. Here, we propose the use of Large Language Models (LLMs) for contextualizing Bayesian optimization (BO) via a hybrid optimization framework that intelligently and economically blends stochastic inference with domain knowledge-based insights from the LLM, which is used to suggest new, better-performing areas of the search space for exploration. Our method fosters user engagement by offering real-time commentary on the optimization progress, explaining the reasoning behind the search strategies. We validate the effectiveness of our approach on synthetic benchmarks with up to 15 independent variables and demonstrate the ability of LLMs to reason in four real-world experimental tasks where context-aware suggestions boost optimization performance substantially.