mlsanityData to 14 Oct 2026

Learning Shared Safety Constraints from Multi-task Demonstrations

arXiv 2309.00711 ↗First posted 1 Sep 2023cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Regardless of the particular task we want them to perform in an environment, there are often shared safety constraints we want our agents to respect. For example, regardless of whether it is making a sandwich or clearing the table, a kitchen robot should not break a plate. Manually specifying such a constraint can be both time-consuming and error-prone. We show how to learn constraints from expert demonstrations of safe task completion by extending inverse reinforcement learning (IRL) techniques to the space of constraints. Intuitively, we learn constraints that forbid highly rewarding behavior that the expert could have taken but chose not to. Unfortunately, the constraint learning problem is rather ill-posed and typically leads to overly conservative constraints that forbid all behavior that the expert did not take. We counter this by leveraging diverse demonstrations that naturally occur in multi-task settings to learn a tighter set of constraints. We validate our method with simulation experiments on high-dimensional continuous control tasks.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 1 Sep 2023.

Builds on

25 of 50 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  7. 2017Inverse Reward Designcited by 249
  8. 2020Projection-Based Constrained Policy Optimizationcited by 203
  9. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  10. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  11. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 81
  12. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 71
13 more
  1. 2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 62
  2. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  3. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  4. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  5. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
  6. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 36
  7. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learningcited by 34
  8. 2017Repeated Inverse Reinforcement Learningcited by 29
  9. 2022Minimax Optimal Online Imitation Learning via Replay Estimationcited by 25
  10. 2018Multi-task Maximum Entropy Inverse Reinforcement Learningcited by 20
  11. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 20
  12. 2023Learning Safety Constraints from Demonstrations with Unknown Rewardscited by 19
  13. 2021Maximum Likelihood Constraint Inference from Stochastic Demonstrationscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysiscited by 45
  2. 2024Hybrid Inverse Reinforcement Learningcited by 32
  3. 2024SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systemscited by 14
  4. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  5. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  6. 2025Your Learned Constraint is Secretly a Backward Reachable Tubecited by 3
  7. 2024Multi Task Inverse Reinforcement Learning for Common Sense Rewardcited by 1
  8. 2024Provably Efficient Exploration in Inverse Constrained Reinforcement Learningcited by 1
  9. 2025Learning Neural Control Barrier Functions from Expert Demonstrations using Inverse Constraint Learningcited by 1
  10. 2024Learning Soft Driving Constraints from Vectorized Scene Embeddings while Imitating Expert Trajectoriescited by 0
  11. 2025DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systemscited by 0
  12. 2026Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systemscited by 0
3 more
  1. 2026Multi-Objective Constraint Inference using Inverse reinforcement learningcited by 0
  2. 2026Implicit Safety Alignment from Crowd Preferencescited by 0
  3. 2026Failing Gracefully: Mitigating Impact of Inevitable Robot Failurescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to maintain safety through expert demonstrations in settings with unknown constraints: A Q-learning perspective
  2. 2023Inverse Constraint Learning and Generalization by Transferable Reward Decomposition ↗
  3. 2018Learning Constraints from Demonstrations
  4. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
  5. 2024Safe CoR: A Dual-Expert Approach to Integrating Imitation Learning and Safe Reinforcement Learning Using Constraint Rewards
  6. 2018Constraint Estimation and Derivative-Free Recovery for Robot Learning from Demonstrations
  7. 2022Safe Inverse Reinforcement Learning via Control Barrier Function
  8. 2026OSIL: Learning Offline Safe Imitation Policies with Safety Inferred from Non-preferred Trajectories