# Positive-Unlabeled Constraint Learning for Inferring Nonlinear Continuous Constraints Functions from Expert Demonstrations

arXiv: https://arxiv.org/abs/2408.01622 · HTML: https://mlsanity.com/paper/2408.01622/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 3 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [Learning from positive and unlabeled data: a survey](https://mlsanity.com/paper/1811.04820/) (2018)
- [Inverse Constrained Reinforcement Learning](https://mlsanity.com/paper/2011.09999/) (2020)
- [Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning](https://mlsanity.com/paper/1909.05477/) (2019)
- [Benchmarking Constraint Inference in Inverse Reinforcement Learning](https://mlsanity.com/paper/2206.09670/) (2022)
- [Maximum Likelihood Constraint Inference from Stochastic Demonstrations](https://mlsanity.com/paper/2102.12554/) (2021)
- [Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian](https://mlsanity.com/paper/2108.11623/) (2021)
- [Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning](https://mlsanity.com/paper/2102.08539/) (2021)
- [Learning Constraints from Locally-Optimal Demonstrations under Cost Function Uncertainty](https://mlsanity.com/paper/2001.09336/) (2020)
- [Learning Behavioral Soft Constraints from Demonstrations](https://mlsanity.com/paper/2202.10407/) (2022)

## Abstract (arXiv)

Planning for diverse real-world robotic tasks necessitates to know and write all constraints. However, instances exist where these constraints are either unknown or challenging to specify accurately. A possible solution is to infer the unknown constraints from expert demonstration. This paper presents a novel two-step Positive-Unlabeled Constraint Learning (PUCL) algorithm to infer a continuous constraint function from demonstrations, without requiring prior knowledge of the true constraint parameterization or environmental model as existing works. We treat all data in demonstrations as positive (feasible) data, and learn a control policy to generate potentially infeasible trajectories, which serve as unlabeled data. The proposed two-step learning framework first identifies reliable infeasible data using a distance metric, and secondly learns a binary feasibility classifier (i.e., constraint function) from the feasible demonstrations and reliable infeasible data. The proposed method is flexible to learn complex-shaped constraint boundary and will not mistakenly classify demonstrations as infeasible as previous methods. The effectiveness of the proposed method is verified in four constrained environments, using a networked policy or a dynamical system policy. It successfully infers the continuous nonlinear constraints and outperforms other baseline methods in terms of constraint accuracy and policy safety. This work has been published in IEEE Robotics and Automation Letters (RA-L). Please refer to the final version at https://doi.org/10.1109/LRA.2024.3522756