# Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds

arXiv: https://arxiv.org/abs/2506.21887 · HTML: https://mlsanity.com/paper/2506.21887/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 104 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 30 of 35)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [MARS: Markov Molecular Sampling for Multi-objective Drug Discovery](https://mlsanity.com/paper/2103.10432/) (2021)
- [A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizations](https://mlsanity.com/paper/1805.12168/) (2018)
- [Decoding-Time Language Model Alignment with Multiple Objectives](https://mlsanity.com/paper/2406.18853/) (2024)
- [Tuning Language Models by Proxy](https://mlsanity.com/paper/2401.08565/) (2024)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Batch Active Preference-Based Learning of Reward Functions](https://mlsanity.com/paper/1810.04303/) (2018)
- [An Easy-to-use Real-world Multi-objective Optimization Problem Suite](https://mlsanity.com/paper/2009.12867/) (2020)
- [Asking Easy Questions: A User-Friendly Approach to Active Reward Learning](https://mlsanity.com/paper/1910.04365/) (2019)
- [Learning Reward Functions by Integrating Human Demonstrations and Preferences](https://mlsanity.com/paper/1906.08928/) (2019)
- [An Emulator for Fine-Tuning Large Language Models using Small Language Models](https://mlsanity.com/paper/2310.12962/) (2023)
- [Quantifying Interpretability and Trust in Machine Learning Systems](https://mlsanity.com/paper/1901.08558/) (2019)
- [Learning Multimodal Rewards from Rankings](https://mlsanity.com/paper/2109.12750/) (2021)
- [Aligning Language Models with Demonstrated Feedback](https://mlsanity.com/paper/2406.00888/) (2024)
- [Computer-Aided Multi-Objective Optimization in Small Molecule Discovery](https://mlsanity.com/paper/2210.07209/) (2022)
- [Learning Reward Functions from Scale Feedback](https://mlsanity.com/paper/2110.00284/) (2021)
- [Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction](https://mlsanity.com/paper/1310.0732/) (2013)
- [Multi-Attribute Bayesian Optimization With Interactive Preference Learning](https://mlsanity.com/paper/1911.05934/) (2019)
- [Including Uncertainty when Learning from Human Corrections](https://mlsanity.com/paper/1806.02454/) (2018)
- [The Green Choice: Learning and Influencing Human Decisions on Shared Roads](https://mlsanity.com/paper/1904.02209/) (2019)
- [Multi-Objective Bayesian Optimization with Active Preference Learning](https://mlsanity.com/paper/2311.13460/) (2023)
- [Learning an Urban Air Mobility Encounter Model from Expert Preferences](https://mlsanity.com/paper/1907.05575/) (2019)
- [Pantheon 1.0, a manually verified dataset of globally famous biographies](https://mlsanity.com/paper/1502.07310/) (2015)

## Built on by (most cited first, 2 of 2)

- [ALMo: Interactive Aim-Limit-Defined, Multi-Objective System for Personalized High-Dose-Rate Brachytherapy Treatment Planning and Visualization for Cervical Cancer](https://mlsanity.com/paper/2602.13666/) (2026)
- [Multi-Objective Bayesian Optimization via Adaptive varepsilon-Constraints Decomposition](https://mlsanity.com/paper/2604.15959/) (2026)

## Abstract (arXiv)

High-stakes decision-making involves navigating multiple competing objectives with expensive evaluations. For instance, in brachytherapy, clinicians must balance maximizing tumor coverage (e.g., an aspirational target or soft bound of >95% coverage) against strict organ dose limits (e.g., a non-negotiable hard bound of <601cGy to the bladder). Selecting Pareto-optimal solutions that match implicit preferences is challenging, as exhaustive Pareto frontier exploration is computationally and cognitively prohibitive, necessitating interactive frameworks to guide users. While decision-makers (DMs) often possess domain knowledge to narrow the search via such soft-hard bounds, current methods often lack systematic approaches to iteratively refine these multi-faceted preference structures. Furthermore, DMs often require confidence that they have not overlooked superior alternatives, a paramount necessity in high-stakes scenarios. We present Active-MoSH, an interactive local-global framework designed for this process. Its local component integrates probabilistic preference learning with an active sampling strategy to adaptively refine Pareto subsets while minimizing cognitive burden. To bolster decision confidence, Active-MoSH's global component, C-MoSH, leverages multi-objective sensitivity analysis to identify potentially overlooked, high-value points beyond immediate feedback. We demonstrate Active-MoSH's performance benefits through diverse synthetic and real-world applications. A high-stakes case study with real cervical cancer brachytherapy treatment plans and an image selection user study further validate our hypotheses regarding the framework's ability to improve convergence, enhance DM confidence, and provide expressive preference articulation.