mlsanityData to 7 Oct 2026

Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes

arXiv 2203.11382 ↗First posted 21 Mar 2022cs.LG math.OC stat.MLcited by 41

Not on the map. Why?

Abstract (arXiv)

We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 19 of its 50 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

19 of 50 references.

References · 19

  1. 2018A Tutorial on Bayesian Optimizationcited by 734
  2. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
  3. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
  4. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  5. 2015Bayesian optimization for materials designcited by 78
  6. 2017Preferential Bayesian Optimizationcited by 72
  7. 2020An Easy-to-use Real-world Multi-objective Optimization Problem Suitecited by 71
  8. 2019Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variablescited by 64
  9. 2019Bayesian Optimization of Composite Functionscited by 45
  10. 2020Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescited by 41
  11. 2021Bayesian Optimization of Function Networkscited by 33
  12. 2022Thinking inside the box: A tutorial on grey-box Bayesian optimizationcited by 26
7 more
  1. 2019Multi-Attribute Bayesian Optimization With Interactive Preference Learningcited by 21
  2. 2020Preferential Batch Bayesian Optimizationcited by 18
  3. 2019Multi-objective Bayesian optimisation with preferences over objectivescited by 16
  4. 2020Real-world Video Adaptation with Reinforcement Learningcited by 12
  5. 2021Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimizationcited by 6
  6. 2018User preferences in Bayesian multi-objective optimization: the expected weighted hypervolume improvement criterioncited by 4
  7. 2016Interactive Preference Learning of Utility Functions for Multi-Objective Optimizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 4; 2025: 10; 2026: 12.

41 papers.

Sort:

By area:

  1. 2022Multi-Objective GFlowNetscited by 100
  2. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 40
  3. 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
  4. 2023Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oraclescited by 34
  5. 2023Looping in the Human Collaborative and Explainable Bayesian Optimizationcited by 27
  6. 2023qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimizationcited by 23
  7. 2023Multi-Objective Bayesian Optimization with Active Preference Learningcited by 13
  8. 2025ComPO: Preference Alignment via Comparison Oraclescited by 11
  9. 2025PABBO: Preferential Amortized Black-Box Optimizationcited by 7
  10. 2025Collaborative Bayesian Optimization via Wasserstein Barycenterscited by 7
  11. 2025User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimizationcited by 6
  12. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
29 more
  1. 2023Preference-Aware Constrained Multi-Objective Bayesian Optimizationcited by 4
  2. 2023Multi-objective optimisation via the R2 utilitiescited by 3
  3. 2023Optimizing Algorithms From Pairwise User Preferencescited by 3
  4. 2024Preferential Multi-Objective Bayesian Optimizationcited by 3
  5. 2026Conditioning Gaussian Processes on Almost Anythingcited by 3
  6. 2022Contextual bandits with concave rewards, and an application to fair rankingcited by 2
  7. 2025Comparative Explanations: Explanation Guided Decision Making for Human-in-the-Loop Preference Selectioncited by 2
  8. 2025Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experimentscited by 2
  9. 2025Bayesian preference elicitation for decision support in multiobjective optimizationcited by 2
  10. 2025Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemblecited by 1
  11. 2025Exploiting Prior Knowledge in Preferential Learning of Individualized Autonomous Vehicle Driving Stylescited by 1
  12. 2025An Interactive Framework for Finding the Optimal Trade-off in Differential Privacycited by 1
  13. 2026Knowledge Gradient for Preference Learningcited by 1
  14. 2026SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testingcited by 1
  15. 2022Efficient Contextual Preferential Bayesian Optimization with Historical Examplescited by 0
  16. 2023The Impact of Expertise in the Loop for Exploring Machine Rationalitycited by 0
  17. 2023Response Time Improves Choice Prediction and Function Estimation for Gaussian Process Models of Perception and Preferencescited by 0
  18. 2024Integrating Human Expertise in Continuous Spaces: A Novel Interactive Bayesian Optimization Framework with Preference Expected Improvementcited by 0
  19. 2024Poisson Process for Bayesian Optimizationcited by 0
  20. 2024Vector Optimization with Gaussian Process Banditscited by 0
  21. 2026Curiosity is Knowledge: Self-Consistent Learning and No-Regret Optimization with Active Inferencecited by 0
  22. 2026Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inferencecited by 0
  23. 2026Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Modelscited by 0
  24. 2026Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedbackcited by 0
  25. 2026Elicitation-Augmented Bayesian Optimizationcited by 0
  26. 2026Local Preferential Bayesian Optimizationcited by 0
  27. 2026Adaptive KappaSharp: Condition-Number Shaping for Preferential Bayesian Optimizationcited by 0
  28. 2026Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Valuescited by 0
  29. 2026A Zeroth-Order Paradigm for LLM Preference Alignmentcited by 0