# Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit

arXiv: https://arxiv.org/abs/2311.14003 · HTML: https://mlsanity.com/paper/2311.14003/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 17 of 17)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Preference-based Online Learning with Dueling Bandits: A Survey](https://mlsanity.com/paper/1807.11398/) (2018)
- [Inverse Preference Learning: Preference-based RL without a Reward Function](https://mlsanity.com/paper/2305.15363/) (2023)
- [Regret Lower Bound and Optimal Algorithm in Dueling Bandit Problem](https://mlsanity.com/paper/1506.02550/) (2015)
- [Double Thompson Sampling for Dueling Bandits](https://mlsanity.com/paper/1604.07101/) (2016)
- [Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Bandits](https://mlsanity.com/paper/1507.04523/) (2015)
- [qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization](https://mlsanity.com/paper/2303.15746/) (2023)
- [Preferential Batch Bayesian Optimization](https://mlsanity.com/paper/2003.11435/) (2020)
- [Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making](https://mlsanity.com/paper/1802.07606/) (2018)
- [Human Preferences as Dueling Bandits](https://mlsanity.com/paper/2204.10362/) (2022)
- [Active Reward Learning from Online Preferences](https://mlsanity.com/paper/2302.13507/) (2023)
- [Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multi-Objective Optimisation Using Reference Points](https://mlsanity.com/paper/1909.13567/) (2019)
- [Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lens](https://mlsanity.com/paper/1309.6830/) (2013)
- [Interactive Decomposition Multi-Objective Optimization via Progressively Learned Value Functions](https://mlsanity.com/paper/1801.00609/) (2018)
- [Interactive Evolutionary Multi-Objective Optimization via Learning-to-Rank](https://mlsanity.com/paper/2204.02604/) (2022)
- [Learning to Actively Learn: A Robust Approach](https://mlsanity.com/paper/2010.15382/) (2020)

## Abstract (arXiv)

Optimization problems find widespread use in both single-objective and multi-objective scenarios. In practical applications, users aspire for solutions that converge to the region of interest (ROI) along the Pareto front (PF). While the conventional approach involves approximating a fitness function or an objective function to reflect user preferences, this paper explores an alternative avenue. Specifically, we aim to discover a method that sidesteps the need for calculating the fitness function, relying solely on human feedback. Our proposed approach entails conducting direct preference learning facilitated by an active dueling bandit algorithm. The experimental phase is structured into three sessions. Firstly, we assess the performance of our active dueling bandit algorithm. Secondly, we implement our proposed method within the context of Multi-objective Evolutionary Algorithms (MOEAs). Finally, we deploy our method in a practical problem, specifically in protein structure prediction (PSP). This research presents a novel interactive preference-based MOEA framework that not only addresses the limitations of traditional techniques but also unveils new possibilities for optimization problems.