# Multi-Objective Controller Synthesis with Uncertain Human Preferences

arXiv: https://arxiv.org/abs/2105.04662 · HTML: https://mlsanity.com/paper/2105.04662/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 8 of 8)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Specifying User Preferences using Weighted Signal Temporal Logic](https://mlsanity.com/paper/2010.00752/) (2020)
- [Permissive Controller Synthesis for Probabilistic Systems](https://mlsanity.com/paper/1504.04662/) (2015)
- [Simple Strategies in Multi-Objective MDPs (Technical Report)](https://mlsanity.com/paper/1910.11024/) (2019)
- [Scenario-Based Verification of Uncertain MDPs](https://mlsanity.com/paper/1912.11223/) (2019)
- [Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes](https://mlsanity.com/paper/1706.06875/) (2017)
- [Farkas certificates and minimal witnesses for probabilistic reachability constraints](https://mlsanity.com/paper/1910.10636/) (2019)
- [Counterexamples for Robotic Planning Explained in Structured Language](https://mlsanity.com/paper/1803.08966/) (2018)

## Built on by (most cited first, 1 of 1)

- [Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guarantees](https://mlsanity.com/paper/2606.29681/) (2026)

## Abstract (arXiv)

Complex real-world applications of cyber-physical systems give rise to the need for multi-objective controller synthesis, which concerns the problem of computing an optimal controller subject to multiple (possibly conflicting) criteria. The relative importance of objectives is often specified by human decision-makers. However, there is inherent uncertainty in human preferences (e.g., due to artifacts resulting from different preference elicitation methods). In this paper, we formalize the notion of uncertain human preferences and present a novel approach that accounts for this uncertainty in the context of multi-objective controller synthesis for Markov decision processes (MDPs). Our approach is based on mixed-integer linear programming and synthesizes an optimally permissive multi-strategy that satisfies uncertain human preferences with respect to a multi-objective property. Experimental results on a range of large case studies show that the proposed approach is feasible and scalable across varying MDP model sizes and uncertainty levels of human preferences. Evaluation via an online user study also demonstrates the quality and benefits of the synthesized controllers.