# Safe Exploration of State and Action Spaces in Reinforcement Learning

arXiv: https://arxiv.org/abs/1402.0560 · HTML: https://mlsanity.com/paper/1402.0560/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2014.

## Built on by (most cited first, 30 of 38)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning](https://mlsanity.com/paper/2106.04480/) (2021)
- [A Provably-Efficient Model-Free Algorithm for Constrained Markov Decision Processes](https://mlsanity.com/paper/2106.01577/) (2021)
- [Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction](https://mlsanity.com/paper/2209.13308/) (2022)
- [A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors](https://mlsanity.com/paper/2302.13137/) (2023)
- [Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications](https://mlsanity.com/paper/2404.09080/) (2024)
- [Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control](https://mlsanity.com/paper/2109.08947/) (2021)
- [Learning Human-like Representations to Enable Learning Human Values](https://mlsanity.com/paper/2312.14106/) (2023)
- [Understanding and Avoiding AI Failures: A Practical Guide](https://mlsanity.com/paper/2104.12582/) (2021)
- [HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents](https://mlsanity.com/paper/2503.08241/) (2025)
- [Safe Deep RL in 3D Environments using Human Feedback](https://mlsanity.com/paper/2201.08102/) (2022)
- [Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2210.03729/) (2022)
- [Model Based Reinforcement Learning for Personalized Heparin Dosing](https://mlsanity.com/paper/2304.10000/) (2023)
- [Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning](https://mlsanity.com/paper/2408.14063/) (2024)
- [Safe Online Bid Optimization with Return on Investment and Budget Constraints](https://mlsanity.com/paper/2201.07139/) (2022)
- [Zero-Shot Uncertainty-Aware Deployment of Simulation Trained Policies on Real-World Robots](https://mlsanity.com/paper/2112.05299/) (2021)
- [Robust AI Driving Strategy for Autonomous Vehicles](https://mlsanity.com/paper/2207.07829/) (2022)
- [Heterogeneous Knowledge for Augmented Modular Reinforcement Learning](https://mlsanity.com/paper/2306.01158/) (2023)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)
- [Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis](https://mlsanity.com/paper/2312.11314/) (2023)
- [Cyber Physical Games](https://mlsanity.com/paper/2407.05817/) (2024)
- [Constraints-of-Thought: A Framework for Constrained Reasoning in Language-Model-Guided Search](https://mlsanity.com/paper/2510.08992/) (2025)
- [SCOPE: Safe Exploration for Dynamic Computer Systems Optimization](https://mlsanity.com/paper/2204.10451/) (2022)
- [Don't do it: Safer Reinforcement Learning With Rule-based Guidance](https://mlsanity.com/paper/2212.13819/) (2022)
- [Imitating careful experts to avoid catastrophic events](https://mlsanity.com/paper/2302.01193/) (2023)
- [Constrained Exploration in Reinforcement Learning with Optimality Preservation](https://mlsanity.com/paper/2304.03104/) (2023)
- [Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)](https://mlsanity.com/paper/2307.04927/) (2023)
- [How to ensure a safe control strategy? Towards a SRL for urban transit autonomous operation](https://mlsanity.com/paper/2311.14457/) (2023)

## Abstract (arXiv)

In this paper, we consider the important problem of safe exploration in reinforcement learning. While reinforcement learning is well-suited to domains with complex transition dynamics and high-dimensional state-action spaces, an additional challenge is posed by the need for safe and efficient exploration. Traditional exploration techniques are not particularly useful for solving dangerous tasks, where the trial and error process may lead to the selection of actions whose execution in some states may result in damage to the learning system (or any other system). Consequently, when an agent begins an interaction with a dangerous and high-dimensional state-action space, an important question arises; namely, that of how to avoid (or at least minimize) damage caused by the exploration of the state-action space. We introduce the PI-SRL algorithm which safely improves suboptimal albeit robust behaviors for continuous state and action control tasks and which efficiently learns from the experience gained from the environment. We evaluate the proposed method in four complex tasks: automatic car parking, pole-balancing, helicopter hovering, and business management.