mlsanityData to 7 Oct 2026

Imitating careful experts to avoid catastrophic events

arXiv 2302.01193 ↗First posted 2 Feb 2023cs.LG cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

RL is increasingly being used to control robotic systems that interact closely with humans. This interaction raises the problem of safe RL: how to ensure that a RL-controlled robotic system never, for instance, injures a human. This problem is especially challenging in rich, realistic settings where it is not even possible to clearly write down a reward function which incorporates these outcomes. In these circumstances, perhaps the only viable approach is based on IRL, which infers rewards from human demonstrations. However, IRL is massively underdetermined as many different rewards can lead to the same optimal policies; we show that this makes it difficult to distinguish catastrophic outcomes (such as injuring a human) from merely undesirable outcomes. Our key insight is that humans do display different behaviour when catastrophic outcomes are possible: they become much more careful. We incorporate carefulness signals into IRL, and find that they do indeed allow IRL to disambiguate undesirable from catastrophic outcomes, which is critical to ensuring safety in future real-world human-robot interactions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

14 of 31 references.

References · 14

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2017Safe Reinforcement Learning via Shieldingcited by 427
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  6. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  7. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  8. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  9. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  10. 2014Safe Exploration of State and Action Spaces in Reinforcement Learningcited by 37
  11. 2019Risk-Aware Active Inverse Reinforcement Learningcited by 26
  12. 2017Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learningcited by 17
2 more
  1. 2017DropoutDAgger: A Bayesian Approach to Safe Imitation Learningcited by 14
  2. 2018One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RLcited by 11

Built on by

Not cited yet.