# Learning Transferable Domain Priors for Safe Exploration in Reinforcement Learning

arXiv: https://arxiv.org/abs/1909.04307 · HTML: https://mlsanity.com/paper/1909.04307/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 7 of 7)

- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain](https://mlsanity.com/paper/2109.06668/) (2021)
- [Reinforcement Learning by Guided Safe Exploration](https://mlsanity.com/paper/2307.14316/) (2023)
- [Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems](https://mlsanity.com/paper/2404.15199/) (2024)
- [Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning](https://mlsanity.com/paper/2405.19414/) (2024)
- [Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learning](https://mlsanity.com/paper/2503.10318/) (2025)
- [Learning to Compress Graphs via Dual Agents for Consistent Topological Robustness Evaluation](https://mlsanity.com/paper/2511.18958/) (2025)

## Abstract (arXiv)

Prior access to domain knowledge could significantly improve the performance of a reinforcement learning agent. In particular, it could help agents avoid potentially catastrophic exploratory actions, which would otherwise have to be experienced during learning. In this work, we identify consistently undesirable actions in a set of previously learned tasks, and use pseudo-rewards associated with them to learn a prior policy. In addition to enabling safer exploratory behaviors in subsequent tasks in the domain, we show that these priors are transferable to similar environments, and can be learned off-policy and in parallel with the learning of other tasks in the domain. We compare our approach to established, state-of-the-art algorithms in both discrete as well as continuous environments, and demonstrate that it exhibits a safer exploratory behavior while learning to perform arbitrary tasks in the domain. We also present a theoretical analysis to support these results, and briefly discuss the implications and some alternative formulations of this approach, which could also be useful in certain scenarios.