# Inverse Constrained Reinforcement Learning

arXiv: https://arxiv.org/abs/2011.09999 · HTML: https://mlsanity.com/paper/2011.09999/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 30 of 46)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Sustainable Online Reinforcement Learning for Auto-bidding](https://mlsanity.com/paper/2210.07006/) (2022)
- [Learning Safety Constraints from Demonstrations with Unknown Rewards](https://mlsanity.com/paper/2305.16147/) (2023)
- [Benchmarking Constraint Inference in Inverse Reinforcement Learning](https://mlsanity.com/paper/2206.09670/) (2022)
- [Identifiability and Generalizability in Constrained Inverse Reinforcement Learning](https://mlsanity.com/paper/2306.00629/) (2023)
- [Learning Soft Constraints From Constrained Expert Demonstrations](https://mlsanity.com/paper/2206.01311/) (2022)
- [An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradient](https://mlsanity.com/paper/2307.08873/) (2023)
- [A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges](https://mlsanity.com/paper/2409.07569/) (2024)
- [Making Human-Like Trade-offs in Constrained Environments by Learning from Demonstrations](https://mlsanity.com/paper/2109.11018/) (2021)
- [Maximum Causal Entropy Inverse Constrained Reinforcement Learning](https://mlsanity.com/paper/2305.02857/) (2023)
- [A Simple Mixture Policy Parameterization for Improving Sample Efficiency of CVaR Optimization](https://mlsanity.com/paper/2403.11062/) (2024)
- [Modeling Others' Minds as Code](https://mlsanity.com/paper/2510.01272/) (2025)
- [Confidence Aware Inverse Constrained Reinforcement Learning](https://mlsanity.com/paper/2406.16782/) (2024)
- [Safety through feedback in Constrained RL](https://mlsanity.com/paper/2406.19626/) (2024)
- [An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2409.09958/) (2024)
- [X-MEN: Guaranteed XOR-Maximum Entropy Constrained Inverse Reinforcement Learning](https://mlsanity.com/paper/2203.11842/) (2022)
- [TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets](https://mlsanity.com/paper/2212.02125/) (2022)
- [Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare](https://mlsanity.com/paper/2410.07525/) (2024)
- [Formalizing the Problem of Side Effect Regularization](https://mlsanity.com/paper/2206.11812/) (2022)
- [Situated Instruction Following](https://mlsanity.com/paper/2407.12061/) (2024)
- [How To Not Drive: Learning Driving Constraints from Demonstration](https://mlsanity.com/paper/2110.00645/) (2021)
- [Learning Behavioral Soft Constraints from Demonstrations](https://mlsanity.com/paper/2202.10407/) (2022)
- [Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learning](https://mlsanity.com/paper/2502.10431/) (2025)
- [Inverse Reinforcement Learning With Constraint Recovery](https://mlsanity.com/paper/2305.08130/) (2023)
- [Multi Task Inverse Reinforcement Learning for Common Sense Reward](https://mlsanity.com/paper/2402.11367/) (2024)
- [Imitating Cost-Constrained Behaviors in Reinforcement Learning](https://mlsanity.com/paper/2403.17456/) (2024)
- [Provably Efficient Exploration in Inverse Constrained Reinforcement Learning](https://mlsanity.com/paper/2409.15963/) (2024)
- [Learning Neural Control Barrier Functions from Expert Demonstrations using Inverse Constraint Learning](https://mlsanity.com/paper/2510.21560/) (2025)
- [Learning Safety Constraints From Demonstration Using One-Class Decision Trees](https://mlsanity.com/paper/2312.08837/) (2023)
- [Safe Reinforcement Learning with Learned Non-Markovian Safety Constraints](https://mlsanity.com/paper/2405.03005/) (2024)

## Abstract (arXiv)

In real world settings, numerous constraints are present which are hard to specify mathematically. However, for the real world deployment of reinforcement learning (RL), it is critical that RL agents are aware of these constraints, so that they can act safely. In this work, we consider the problem of learning constraints from demonstrations of a constraint-abiding agent's behavior. We experimentally validate our approach and show that our framework can successfully learn the most likely constraints that the agent respects. We further show that these learned constraints are textittransferable to new agents that may have different morphologies and/or reward functions. Previous works in this regard have either mainly been restricted to tabular (discrete) settings, specific types of constraints or assume the environment's transition dynamics. In contrast, our framework is able to learn arbitrary textitMarkovian constraints in high-dimensions in a completely model-free setting. The code can be found it: urlhttps://github.com/shehryar-malik/icrl.