# A Safe Preference Learning Approach for Personalization with Applications to Autonomous Vehicles

arXiv: https://arxiv.org/abs/2311.02099 · HTML: https://mlsanity.com/paper/2311.02099/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Active Preference-Based Learning of Reward Functions](https://mlsanity.com/paper/1810.04303/) (2018)
- [Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods](https://mlsanity.com/paper/2008.00097/) (2020)
- [Learning Linear Temporal Properties](https://mlsanity.com/paper/1806.03953/) (2018)
- [Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks](https://mlsanity.com/paper/2007.01498/) (2020)
- [On Robustness Metrics for Learning STL Tasks](https://mlsanity.com/paper/2003.06041/) (2020)
- [Specifying User Preferences using Weighted Signal Temporal Logic](https://mlsanity.com/paper/2010.00752/) (2020)
- [A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks](https://mlsanity.com/paper/1709.09611/) (2017)
- [A First Look at Ad-block Detection: A New Arms Race on the Web](https://mlsanity.com/paper/1605.05841/) (2016)
- [Information-Guided Temporal Logic Inference with Prior Knowledge](https://mlsanity.com/paper/1811.08846/) (2018)
- [Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations](https://mlsanity.com/paper/2006.02411/) (2020)
- [Learning Signal Temporal Logic through Neural Network for Interpretable Classification](https://mlsanity.com/paper/2210.01910/) (2022)
- [Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots](https://mlsanity.com/paper/2011.05424/) (2020)
- [Safety-Aware Preference-Based Learning for Safety-Critical Control](https://mlsanity.com/paper/2112.08516/) (2021)
- [Differentiable Inference of Temporal Logic Formulas](https://mlsanity.com/paper/2208.05440/) (2022)
- [Sampling-Based Optimal Control Synthesis for Multi-Robot Systems under Global Temporal Tasks](https://mlsanity.com/paper/1706.04216/) (2017)
- [Learning Performance Graphs from Demonstrations via Task-Based Evaluations](https://mlsanity.com/paper/2204.05909/) (2022)
- [Temporal Logic Guided Motion Primitives for Complex Manipulation Tasks with User Preferences](https://mlsanity.com/paper/2202.04375/) (2022)

## Built on by (most cited first, 1 of 1)

- [Coloring Between the Lines: Personalization in the Null Space of Planning Constraints](https://mlsanity.com/paper/2505.15503/) (2025)

## Abstract (arXiv)

This work introduces a preference learning method that ensures adherence to given specifications, with an application to autonomous vehicles. Our approach incorporates the priority ordering of Signal Temporal Logic (STL) formulas describing traffic rules into a learning framework. By leveraging Parametric Weighted Signal Temporal Logic (PWSTL), we formulate the problem of safety-guaranteed preference learning based on pairwise comparisons and propose an approach to solve this learning problem. Our approach finds a feasible valuation for the weights of the given PWSTL formula such that, with these weights, preferred signals have weighted quantitative satisfaction measures greater than their non-preferred counterparts. The feasible valuation of weights given by our approach leads to a weighted STL formula that can be used in correct-and-custom-by-construction controller synthesis. We demonstrate the performance of our method with a pilot human subject study in two different simulated driving scenarios involving a stop sign and a pedestrian crossing. Our approach yields competitive results compared to existing preference learning methods in terms of capturing preferences and notably outperforms them when safety is considered.