# Multi-Attribute Bayesian Optimization With Interactive Preference Learning

arXiv: https://arxiv.org/abs/1911.05934 · HTML: https://mlsanity.com/paper/1911.05934/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 21 of 21)

- [Pareto Set Learning for Expensive Multi-Objective Optimization](https://mlsanity.com/paper/2210.08495/) (2022)
- [Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes](https://mlsanity.com/paper/2203.11382/) (2022)
- [Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles](https://mlsanity.com/paper/2303.03751/) (2023)
- [Looping in the Human Collaborative and Explainable Bayesian Optimization](https://mlsanity.com/paper/2310.17273/) (2023)
- [Multi-Objective Bayesian Optimization with Active Preference Learning](https://mlsanity.com/paper/2311.13460/) (2023)
- [Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2406.08799/) (2024)
- [ComPO: Preference Alignment via Comparison Oracles](https://mlsanity.com/paper/2505.05465/) (2025)
- [User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2502.06971/) (2025)
- [Two-step Lookahead Bayesian Optimization with Inequality Constraints](https://mlsanity.com/paper/2112.02833/) (2021)
- [Multi-objective optimisation via the R2 utilities](https://mlsanity.com/paper/2305.11774/) (2023)
- [Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties](https://mlsanity.com/paper/2402.17343/) (2024)
- [Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds](https://mlsanity.com/paper/2506.21887/) (2025)
- [Bayesian preference elicitation for decision support in multiobjective optimization](https://mlsanity.com/paper/2507.16999/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [One Step Preference Elicitation in Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2105.13278/) (2021)
- [Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble](https://mlsanity.com/paper/2501.18792/) (2025)
- [An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy](https://mlsanity.com/paper/2509.04290/) (2025)
- [Vector Optimization with Gaussian Process Bandits](https://mlsanity.com/paper/2412.02484/) (2024)
- [Active Preference Learning over Latent Preference Archetypes for Many-Objective Bayesian Optimization](https://mlsanity.com/paper/2603.28410/) (2026)
- [Elicitation-Augmented Bayesian Optimization](https://mlsanity.com/paper/2605.12079/) (2026)
- [A Zeroth-Order Paradigm for LLM Preference Alignment](https://mlsanity.com/paper/2609.19144/) (2026)

## Abstract (arXiv)

We consider black-box global optimization of time-consuming-to-evaluate functions on behalf of a decision-maker (DM) whose preferences must be learned. Each feasible design is associated with a time-consuming-to-evaluate vector of attributes and each vector of attributes is assigned a utility by the DM's utility function, which may be learned approximately using preferences expressed over pairs of attribute vectors. Past work has used a point estimate of this utility function as if it were error-free within single-objective optimization. However, utility estimation errors may yield a poor suggested design. Furthermore, this approach produces a single suggested "best" design, whereas DMs often prefer to choose from a menu. We propose a novel multi-attribute Bayesian optimization with preference learning approach. Our approach acknowledges the uncertainty in preference estimation and implicitly chooses designs to evaluate that are good not just for a single estimated utility function but a range of likely ones. The outcome of our approach is a menu of designs and evaluated attributes from which the DM makes a final selection. We demonstrate the value and flexibility of our approach in a variety of experiments.