# SemanticOpt: Towards LLM-Based Semantic Black-Box Optimization

arXiv: https://arxiv.org/abs/2510.25404 · HTML: https://mlsanity.com/paper/2510.25404/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 26 of 26)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Unexpected Improvements to Expected Improvement for Bayesian Optimization](https://mlsanity.com/paper/2310.20708/) (2023)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Large Language Models As Evolution Strategies](https://mlsanity.com/paper/2402.18381/) (2024)
- [End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes](https://mlsanity.com/paper/2305.15930/) (2023)
- [Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models](https://mlsanity.com/paper/2404.06290/) (2024)
- [Reinforced In-Context Black-Box Optimization](https://mlsanity.com/paper/2402.17423/) (2024)
- [The Vizier Gaussian Process Bandit Algorithm](https://mlsanity.com/paper/2408.11527/) (2024)
- [LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization](https://mlsanity.com/paper/2505.14756/) (2025)
- [OptunaHub: A Platform for Black-Box Optimization](https://mlsanity.com/paper/2510.02798/) (2025)
- [OpenBox: A Python Toolkit for Generalized Black-box Optimization](https://mlsanity.com/paper/2304.13339/) (2023)
- [Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMs](https://mlsanity.com/paper/2505.12833/) (2025)
- [AutoOED: Automated Optimal Experiment Design Platform](https://mlsanity.com/paper/2104.05959/) (2021)
- [Pre-trained knowledge elevates large language models beyond traditional chemical reaction optimizers](https://mlsanity.com/paper/2509.00103/) (2025)
- [ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization](https://mlsanity.com/paper/2510.03051/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization](https://mlsanity.com/paper/2610.12183/) (2026)

## Abstract (arXiv)

Optimizing an experimental system can be extremely challenging when each experiment is expensive, time-consuming, or difficult to perform. Existing optimizers for expensive black-box problems, such as Bayesian optimization, are typically limited to numerical or categorical observations. They do not make use of broader domain knowledge, such as expert heuristics, relevant scientific papers, or similar previous experiments. Large language models (LLMs) can interpret this semantic information; however, even state-of-the-art LLMs struggle to reliably solve black-box optimization problems. We introduce SemanticOpt, a framework for semantic black-box optimization that equips LLMs with optimization capabilities by fine-tuning them on structured Bayesian optimization trajectories augmented with natural-language context. SemanticOpt jointly uses numerical and semantic evidence when proposing new experiments, while producing interpretable predictions aligned with Bayesian surrogate models. We construct a range of real-world optimization problems paired with semantic information to create a diverse benchmark for evaluating semantic black-box optimization. Across these domains, SemanticOpt outperforms both classical optimizers and existing LLM-based approaches on average when given relevant semantic information.