# Safety-Aware Apprenticeship Learning

arXiv: https://arxiv.org/abs/1710.07983 · HTML: https://mlsanity.com/paper/1710.07983/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2017.

## Built on by (most cited first, 12 of 12)

- [Learning from Demonstrations using Signal Temporal Logic](https://mlsanity.com/paper/2102.07730/) (2021)
- [Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities](https://mlsanity.com/paper/2305.17568/) (2023)
- [A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety](https://mlsanity.com/paper/2505.17342/) (2025)
- [Programmatic Reward Design by Example](https://mlsanity.com/paper/2112.08438/) (2021)
- [A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machines](https://mlsanity.com/paper/2204.09772/) (2022)
- [Learning Performance Graphs from Demonstrations via Task-Based Evaluations](https://mlsanity.com/paper/2204.05909/) (2022)
- [Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment](https://mlsanity.com/paper/2410.23680/) (2024)
- [Signal Temporal Logic-Guided Apprenticeship Learning](https://mlsanity.com/paper/2311.05084/) (2023)
- [Safety-Aware Multi-Agent Apprenticeship Learning](https://mlsanity.com/paper/2201.08111/) (2022)
- [PAGAR: Taming Reward Misalignment in Inverse Reinforcement Learning-Based Imitation Learning with Protagonist Antagonist Guided Adversarial Reward](https://mlsanity.com/paper/2306.01731/) (2023)
- [Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)](https://mlsanity.com/paper/2307.04927/) (2023)
- [Learning from Imperfect Demonstrations via Temporal Behavior Tree-Guided Trajectory Repair](https://mlsanity.com/paper/2604.04225/) (2026)

## Abstract (arXiv)

Apprenticeship learning (AL) is a kind of Learning from Demonstration techniques where the reward function of a Markov Decision Process (MDP) is unknown to the learning agent and the agent has to derive a good policy by observing an expert's demonstrations. In this paper, we study the problem of how to make AL algorithms inherently safe while still meeting its learning objective. We consider a setting where the unknown reward function is assumed to be a linear combination of a set of state features, and the safety property is specified in Probabilistic Computation Tree Logic (PCTL). By embedding probabilistic model checking inside AL, we propose a novel counterexample-guided approach that can ensure safety while retaining performance of the learnt policy. We demonstrate the effectiveness of our approach on several challenging AL scenarios where safety is essential.