# Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMs

arXiv: https://arxiv.org/abs/2505.12833 · HTML: https://mlsanity.com/paper/2505.12833/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 33)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Qwen2.5-1M Technical Report](https://mlsanity.com/paper/2501.15383/) (2025)
- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework](https://mlsanity.com/paper/2305.03268/) (2023)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning](https://mlsanity.com/paper/2503.05379/) (2025)
- [CMA-ES for Hyperparameter Optimization of Deep Neural Networks](https://mlsanity.com/paper/1604.07269/) (2016)
- [Scalable Pareto Front Approximation for Deep Multi-Objective Learning](https://mlsanity.com/paper/2103.13392/) (2021)
- [Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning](https://mlsanity.com/paper/2502.19655/) (2025)
- [Unveiling the Generalization Power of Fine-Tuned Large Language Models](https://mlsanity.com/paper/2403.09162/) (2024)
- [Time Efficiency in Optimization with a Bayesian-Evolutionary Algorithm](https://mlsanity.com/paper/2005.04166/) (2020)
- [Sequential Large Language Model-Based Hyper-parameter Optimization](https://mlsanity.com/paper/2410.20302/) (2024)
- [A Bayesian approach to constrained single- and multi-objective optimization](https://mlsanity.com/paper/1510.00503/) (2015)
- [A General Framework for User-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.14645/) (2023)
- [Policy Gradient Methods for Reinforcement Learning with Function Approximation and Action-Dependent Baselines](https://mlsanity.com/paper/1706.06643/) (2017)
- [Language-Based Bayesian Optimization Research Assistant (BORA)](https://mlsanity.com/paper/2501.16224/) (2025)
- [Bayesian optimization with known experimental and design constraints for chemistry applications](https://mlsanity.com/paper/2203.17241/) (2022)
- [Approximation-Aware Bayesian Optimization](https://mlsanity.com/paper/2406.04308/) (2024)

## Built on by (most cited first, 8 of 8)

- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery](https://mlsanity.com/paper/2605.17976/) (2026)
- [LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation](https://mlsanity.com/paper/2605.22054/) (2026)
- [Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch](https://mlsanity.com/paper/2608.00316/) (2026)
- [ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond](https://mlsanity.com/paper/2608.14354/) (2026)
- [DASH: Decoupled Adaptive Surrogate - Acquisition Harness for Automated Bayesian Optimization](https://mlsanity.com/paper/2608.00641/) (2026)
- [SemanticOpt: Towards LLM-Based Semantic Black-Box Optimization](https://mlsanity.com/paper/2510.25404/) (2025)
- [Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search](https://mlsanity.com/paper/2609.34281/) (2026)

## Abstract (arXiv)

Many real-world scientific and industrial applications require the optimization of expensive black-box functions. Bayesian Optimization (BO) provides an effective framework for such problems. However, traditional BO methods are prone to get trapped in local optima and often lack interpretable insights. To address this issue, this paper designs Reasoning BO, a novel framework that leverages reasoning models to guide the sampling process in BO while incorporating multi-agent systems and knowledge graphs for online knowledge accumulation. By integrating the reasoning and contextual understanding capabilities of Large Language Models (LLMs), we can provide strong guidance to enhance the BO process. As the optimization progresses, Reasoning BO provides real-time sampling recommendations along with critical insights grounded in plausible scientific theories, aiding in the discovery of superior solutions within the search space. We systematically evaluate our approach across 10 diverse tasks encompassing synthetic mathematical functions and complex real-world applications. The framework demonstrates its capability to progressively refine sampling strategies through real-time insights and hypothesis evolution, effectively identifying higher-performing regions of the search space for focused exploration. This process highlights the powerful reasoning and context-learning abilities of LLMs in optimization scenarios. For example, in the Direct Arylation task, our method increased the yield to 60.7%, whereas traditional BO achieved only a 25.2% yield. Furthermore, our investigation reveals that smaller LLMs, when fine-tuned through reinforcement learning, can attain comparable performance to their larger counterparts.