mlsanityData to 14 Oct 2026

Preferential Bayesian Optimization

arXiv 1704.03651 ↗First posted 12 Apr 2017stat.MLcited by 74

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) has emerged during the last few years as an effective approach to optimizing black-box functions where direct queries of the objective are expensive. In this paper we consider the case where direct access to the function is not possible, but information about user preferences is. Such scenarios arise in problems where human preferences are modeled, such as A/B tests or recommender systems. We present a new framework for this scenario that we call Preferential Bayesian Optimization (PBO) which allows us to find the optimum of a latent function that can only be queried through pairwise comparisons, the so-called duels. PBO extends the applicability of standard BO ideas and generalizes previous discrete dueling approaches by modeling the probability of the winner of each duel by means of a Gaussian process model with a Bernoulli likelihood. The latent preference function is used to define a family of acquisition functions that extend usual policies used in BO. We illustrate the benefits of PBO in a variety of experiments, showing that PBO needs drastically fewer comparisons for finding the optimum. According to our experiments, the way of modeling correlations in PBO is key in obtaining this advantage.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 8; 2022: 10; 2023: 9; 2024: 14; 2025: 17; 2026: 15.

74 papers.

Sort:

By area:

  1. 2020Active Preference-Based Gaussian Process Regression for Reward Learningcited by 71
  2. 2021Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizabilitycited by 44
  3. 2022Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomescited by 42
  4. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 41
  5. 2022Stochastic Contextual Dueling Bandits under Linear Stochastic Transitivity Modelscited by 37
  6. 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
  7. 2023Looping in the Human Collaborative and Explainable Bayesian Optimizationcited by 27
  8. 2023qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimizationcited by 23
  9. 2023Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processescited by 19
  10. 2024Bayesian Optimization with LLM-Based Acquisition Functions for Natural Language Preference Elicitationcited by 17
  11. 2024Principled Preferential Bayesian Optimizationcited by 16
  12. 2024Sample-Efficient Alignment for LLMscited by 16
38 more
  1. 2023Multi-Objective Bayesian Optimization with Active Preference Learningcited by 13
  2. 2022Exploiting Correlation to Achieve Faster Learning Rates in Low-Rank Preference Banditscited by 11
  3. 2023Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prosthesescited by 8
  4. 2025PABBO: Preferential Amortized Black-Box Optimizationcited by 8
  5. 2024Bandits with Preference Feedback: A Stackelberg Game Perspectivecited by 7
  6. 2026Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discoverycited by 7
  7. 2021Toward AI Assistants That Let Designers Designcited by 6
  8. 2022Explaining Preferences with Shapley Valuescited by 6
  9. 2022Bayesian Optimization Augmented with Actively Elicited Expert Knowledgecited by 6
  10. 2022Dueling Convex Optimization with General Preferencescited by 6
  11. 2025Inference-Time Personalized Alignment with a Few User Preference Queriescited by 6
  12. 2021Efficient Exploration in Binary and Preferential Bayesian Optimizationcited by 5
  13. 2024Principled Bayesian Optimisation in Collaboration with Human Expertscited by 5
  14. 2025Best Policy Learning from Trajectory Preference Feedbackcited by 5
  15. 2021End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimizationcited by 4
  16. 2023Optimizing Algorithms From Pairwise User Preferencescited by 4
  17. 2024Ranking over Regression for Bayesian Optimization and Molecule Selectioncited by 4
  18. 2024Streaming Bayes GFlowNetscited by 4
  19. 2021On Preference Learning Based on Sequential Bayesian Optimization with Pairwise Comparisoncited by 3
  20. 2021Bayesian Optimisation for Sequential Experimental Design with Applications in Additive Manufacturingcited by 3
  21. 2022GLISp-r: A preference-based optimization algorithm with convergence guaranteescited by 3
  22. 2024Enhanced Bayesian Optimization via Preferential Modeling of Abstract Propertiescited by 3
  23. 2024How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?cited by 3
  24. 2024Preferential Multi-Objective Bayesian Optimizationcited by 3
  25. 2025Bayesian Optimization from Human Feedback: Near-Optimal Regret Boundscited by 3
  26. 2021Choice functions based multi-objective Bayesian optimisationcited by 2
  27. 2024Balancing Optimality and Diversity: Human-Centered Decision Making through Generative Curationcited by 2
  28. 2025Comparative Explanations: Explanation Guided Decision Making for Human-in-the-Loop Preference Selectioncited by 2
  29. 2025Efficient Visual Appearance Optimization by Learning from Prior Preferencescited by 2
  30. 2025Bayesian preference elicitation for decision support in multiobjective optimizationcited by 2
  31. 2025Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectoriescited by 2
  32. 2025Efficient Reinforcement Learning from Human Feedback via Bayesian Preference Inferencecited by 2
  33. 2021Contextual Bayesian optimization with binary outputscited by 1
  34. 2023The Impact of Expertise in the Loop for Exploring Machine Rationalitycited by 1
  35. 2024Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimizationcited by 1
  36. 2025Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemblecited by 1
  37. 2025A Guide to Bayesian Optimization in Bioprocess Engineeringcited by 1
  38. 2025An Interactive Framework for Finding the Optimal Trade-off in Differential Privacycited by 1

50 most cited of 74 All 74 (JSON)