How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
Bayesian optimization (BO) is an integral part of automated scientific discovery -- the so-called self-driving lab -- where human inputs are ideally minimal or at least non-blocking. However, scientists often have strong intuition, and thus human feedback is still useful. Nevertheless, prior works in enhancing BO with expert feedback, such as by incorporating it in an offline or online but blocking (arrives at each BO iteration) manner, are incompatible with the spirit of self-driving labs. In this work, we study whether a small amount of randomly arriving expert feedback that is being incorporated in a non-blocking manner can improve a BO campaign. To this end, we run an additional, independent computing thread on top of the BO loop to handle the feedback-gathering process. The gathered feedback is used to learn a Bayesian preference model that can readily be incorporated into the BO thread, to steer its exploration-exploitation process. Experiments on toy and chemistry datasets suggest that even just a few intermittent, asynchronous expert feedback can be useful for improving or constraining BO. This can especially be useful for its implication in improving self-driving labs, e.g. making them more data-efficient and less costly.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 30 references that are in the map. First posted on arXiv 10 Jun 2024.
Builds on
15 of 30 references.
Closest ancestors · 2
Same direction, newest first
- 2024Bayesian Reward Models for LLM Alignment20 Feb
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
Other references · 13
- 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
- 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
- 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 98
- 2017Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Spacecited by 76
- 2017Preferential Bayesian Optimizationcited by 72
- 2020Accelerating high-throughput virtual screening through molecular pool-based active learningcited by 59
- 2023A Study of Bayesian Neural Network Surrogates for Bayesian Optimizationcited by 50
- 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 46
- 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 40
- 2023Looping in the Human Collaborative and Explainable Bayesian Optimizationcited by 27
- 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 12
- 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
Built on by
Citing papers per year: 2025: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds
- 2026La Agente 'Optima: Towards Agentic Self-Driving Laboratories
- 2025Reasoning BO: Enhancing Bayesian Optimization with Long-Context Reasoning Power of LLMs
- 2026Agentic High-Dimensional Bayesian Optimization with Hypothesis- and Evidence-Guided Search
- 2026Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch
- 2024Preferential Multi-Objective Bayesian Optimization
- 2026The Bayesian Reflex: Online Learning as the Autonomic Nervous System of Modern and Future AI
- 2026Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery