mlsanityData to 13 Oct 2026

Reinforcement Learning by Guided Safe Exploration

arXiv 2307.14316 ↗First posted 26 Jul 2023cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Safety is critical to broadening the application of reinforcement learning (RL). Often, we train RL agents in a controlled environment, such as a laboratory, before deploying them in the real world. However, the real-world target task might be unknown prior to deployment. Reward-free RL trains an agent without the reward to adapt quickly once the reward is revealed. We consider the constrained reward-free setting, where an agent (the guide) learns to explore safely without the reward signal. This agent is trained in a controlled environment, which allows unsafe interactions and still provides the safety signal. After the target task is revealed, safety violations are not allowed anymore. Thus, the guide is leveraged to compose a safe behaviour policy. Drawing from transfer learning, we also regularize a target policy (the student) towards the guide while the student is unreliable and gradually eliminate the influence of the guide as training progresses. The empirical analysis shows that this method can achieve safe transfer learning and helps the student solve the target task faster.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Builds on 41 of its 62 references that are in the map. First posted on arXiv 26 Jul 2023.

Builds on

41 of 62 references.

References · 41

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,870
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,185
  4. 2017Constrained Policy Optimizationcited by 931
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 739
  6. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 660
  7. 2017Safe Reinforcement Learning via Shieldingcited by 431
  8. 2018Reward Constrained Policy Optimizationcited by 400
  9. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  10. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  11. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  12. 2020Projection-Based Constrained Policy Optimizationcited by 196
29 more
  1. 2019Efficient Exploration via State Marginal Matchingcited by 186
  2. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 170
  3. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 157
  4. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 110
  5. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  6. 2015An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learningcited by 103
  7. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  8. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  9. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  10. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  11. 2017Near Optimal Behavior via Approximate State Abstractioncited by 64
  12. 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 51
  13. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  14. 2021A Simple Reward-free Approach to Constrained Reinforcement Learningcited by 29
  15. 2021Safe Driving via Expert Guided Policy Optimizationcited by 29
  16. 2021Density Constrained Reinforcement Learningcited by 25
  17. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  18. 2020Safe Active Dynamics Learning and Control: A Sequential Exploration-Exploitation Frameworkcited by 19
  19. 2021Safe Continuous Control with Constrained Model-Based Policy Optimizationcited by 17
  20. 2019Marginalized State Distribution Entropy Regularization in Policy Optimizationcited by 16
  21. 2021Risk-Aware Transfer in Reinforcement Learning using Successor Featurescited by 12
  22. 2020Accelerating Safe Reinforcement Learning with Constraint-mismatched Policiescited by 11
  23. 2021Curriculum Learning for Safe Mapless Navigationcited by 11
  24. 2019Learning latent state representation for speeding up explorationcited by 8
  25. 2019Learning Transferable Domain Priors for Safe Exploration in Reinforcement Learningcited by 7
  26. 2021MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerancecited by 7
  27. 2020Multiple Plans are Better than One: Diverse Stochastic Planningcited by 5
  28. 2021Maximum Entropy Model-based Reinforcement Learningcited by 5
  29. 2020Safe Reinforcement Learning through Meta-learned Instinctscited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 4.

6 papers.

  1. 2024Safe Multiagent Coordination via Entropic Explorationcited by 3
  2. 2026Neuro-symbolic Action Masking for Deep Reinforcement Learningcited by 1
  3. 2024Model Checking for Reinforcement Learning in Autonomous Driving: One Can Do More Than You Think!cited by 0
  4. 2026SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learningcited by 0
  5. 2026Global Optimality for Constrained Exploration via Penalty Regularizationcited by 0
  6. 2026Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learningcited by 0