# Bayesian Constraint Inference from User Demonstrations Based on Margin-Respecting Preference Models

arXiv: https://arxiv.org/abs/2403.02431 · HTML: https://mlsanity.com/paper/2403.02431/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2024.

## Built on by (most cited first, 4 of 4)

- [A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges](https://mlsanity.com/paper/2409.07569/) (2024)
- [DRIVE: Dynamic Rule Inference and Verified Evaluation for Constraint-Aware Autonomous Driving](https://mlsanity.com/paper/2508.04066/) (2025)
- [Active Constraint Learning in High Dimensions from Demonstrations](https://mlsanity.com/paper/2512.22757/) (2025)
- [Implicit Safety Alignment from Crowd Preferences](https://mlsanity.com/paper/2605.21822/) (2026)

## Abstract (arXiv)

It is crucial for robots to be aware of the presence of constraints in order to acquire safe policies. However, explicitly specifying all constraints in an environment can be a challenging task. State-of-the-art constraint inference algorithms learn constraints from demonstrations, but tend to be computationally expensive and prone to instability issues. In this paper, we propose a novel Bayesian method that infers constraints based on preferences over demonstrations. The main advantages of our proposed approach are that it 1) infers constraints without calculating a new policy at each iteration, 2) uses a simple and more realistic ranking of groups of demonstrations, without requiring pairwise comparisons over all demonstrations, and 3) adapts to cases where there are varying levels of constraint violation. Our empirical results demonstrate that our proposed Bayesian approach infers constraints of varying severity, more accurately than state-of-the-art constraint inference methods.