# Multi-Objective Bayesian Optimization with Active Preference Learning

arXiv: https://arxiv.org/abs/2311.13460 · HTML: https://mlsanity.com/paper/2311.13460/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizations](https://mlsanity.com/paper/1805.12168/) (2018)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraints](https://mlsanity.com/paper/2009.01721/) (2020)
- [Preferential Bayesian Optimization](https://mlsanity.com/paper/1704.03651/) (2017)
- [Multi-objective Bayesian Optimization using Pareto-frontier Entropy](https://mlsanity.com/paper/1906.00127/) (2019)
- [AutoBalance: Optimized Loss Functions for Imbalanced Data](https://mlsanity.com/paper/2201.01212/) (2022)
- [Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes](https://mlsanity.com/paper/2203.11382/) (2022)
- [Multi-Attribute Bayesian Optimization With Interactive Preference Learning](https://mlsanity.com/paper/1911.05934/) (2019)
- [Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes](https://mlsanity.com/paper/2302.01513/) (2023)
- [Multi-objective Bayesian optimisation with preferences over objectives](https://mlsanity.com/paper/1902.04228/) (2019)
- [Preference-Aware Constrained Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2303.13034/) (2023)

## Built on by (most cited first, 13 of 13)

- [User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2502.06971/) (2025)
- [Pareto-frontier Entropy Search with Variational Lower Bound Maximization](https://mlsanity.com/paper/2501.19073/) (2025)
- [Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds](https://mlsanity.com/paper/2506.21887/) (2025)
- [LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection](https://mlsanity.com/paper/2510.25799/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble](https://mlsanity.com/paper/2501.18792/) (2025)
- [From Automation to Autonomy in Smart Manufacturing: A Bayesian Optimization Framework for Modeling Multi-Objective Experimentation and Sequential Decision Making](https://mlsanity.com/paper/2504.04244/) (2025)
- [An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy](https://mlsanity.com/paper/2509.04290/) (2025)
- [NOSTRA: A noise-resilient and sparse data framework for trust region based multi objective Bayesian optimization](https://mlsanity.com/paper/2508.16476/) (2025)
- [Active Preference Learning over Latent Preference Archetypes for Many-Objective Bayesian Optimization](https://mlsanity.com/paper/2603.28410/) (2026)
- [Elicitation-Augmented Bayesian Optimization](https://mlsanity.com/paper/2605.12079/) (2026)
- [Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery](https://mlsanity.com/paper/2608.04651/) (2026)
- [Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation](https://mlsanity.com/paper/2609.13396/) (2026)

## Abstract (arXiv)

There are a lot of real-world black-box optimization problems that need to optimize multiple criteria simultaneously. However, in a multi-objective optimization (MOO) problem, identifying the whole Pareto front requires the prohibitive search cost, while in many practical scenarios, the decision maker (DM) only needs a specific solution among the set of the Pareto optimal solutions. We propose a Bayesian optimization (BO) approach to identifying the most preferred solution in the MOO with expensive objective functions, in which a Bayesian preference model of the DM is adaptively estimated by an interactive manner based on the two types of supervisions called the pairwise preference and improvement request. To explore the most preferred solution, we define an acquisition function in which the uncertainty both in the objective functions and the DM preference is incorporated. Further, to minimize the interaction cost with the DM, we also propose an active learning strategy for the preference estimation. We empirically demonstrate the effectiveness of our proposed method through the benchmark function optimization and the hyper-parameter optimization problems for machine learning models.