# A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors

arXiv: https://arxiv.org/abs/2302.13137 · HTML: https://mlsanity.com/paper/2302.13137/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 90 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 30 of 44)

- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [Supervising strong learners by amplifying weak experts](https://mlsanity.com/paper/1810.08575/) (2018)
- [Lyapunov-based Safe Policy Optimization for Continuous Control](https://mlsanity.com/paper/1901.10031/) (2019)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Collaborating with Humans without Human Data](https://mlsanity.com/paper/2110.08176/) (2021)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Conservative Safety Critics for Exploration](https://mlsanity.com/paper/2010.14497/) (2020)
- [Trial without Error: Towards Safe Reinforcement Learning via Human Intervention](https://mlsanity.com/paper/1707.05173/) (2017)
- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Safe Exploration in Finite Markov Decision Processes with Gaussian Processes](https://mlsanity.com/paper/1606.04753/) (2016)
- [Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions](https://mlsanity.com/paper/2112.11561/) (2021)
- [Safe Exploration for Interactive Machine Learning](https://mlsanity.com/paper/1910.13726/) (2019)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems](https://mlsanity.com/paper/1909.03602/) (2019)
- [Learning Human Objectives by Evaluating Hypothetical Behavior](https://mlsanity.com/paper/1912.05652/) (2019)
- [A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges](https://mlsanity.com/paper/2105.12949/) (2021)
- [Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes](https://mlsanity.com/paper/1603.04915/) (2016)
- [Safe Exploration of State and Action Spaces in Reinforcement Learning](https://mlsanity.com/paper/1402.0560/) (2014)
- [On the Robustness of Safe Reinforcement Learning under Observational Perturbations](https://mlsanity.com/paper/2205.14691/) (2022)
- [Multi-Agent Constrained Policy Optimisation](https://mlsanity.com/paper/2110.02793/) (2021)
- [Verifiably Safe Exploration for End-to-End Reinforcement Learning](https://mlsanity.com/paper/2007.01223/) (2020)

## Built on by (most cited first, 15 of 15)

- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation](https://mlsanity.com/paper/2405.01677/) (2024)
- [Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2405.16390/) (2024)
- [Safe Multi-Agent Reinforcement Learning with Bilevel Optimization in Autonomous Driving](https://mlsanity.com/paper/2405.18209/) (2024)
- [Mutual Enhancement of Large Language and Reinforcement Learning Models through Bi-Directional Feedback Mechanisms: A Planning Case Study](https://mlsanity.com/paper/2401.06603/) (2024)
- [Safe Reinforcement Learning for Power System Control: A Review](https://mlsanity.com/paper/2407.00681/) (2024)
- [Reinforcement Learning and Machine ethics:a systematic review](https://mlsanity.com/paper/2407.02425/) (2024)
- [Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2512.10510/) (2025)
- [OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2605.15971/) (2026)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering](https://mlsanity.com/paper/2606.29201/) (2026)
- [SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization](https://mlsanity.com/paper/2311.00880/) (2023)
- [HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads](https://mlsanity.com/paper/2606.00252/) (2026)
- [Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models](https://mlsanity.com/paper/2607.13172/) (2026)
- [Safe Greenhouse Climate Control Using Lagrangian-Constrained PPO with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2609.34966/) (2026)

## Abstract (arXiv)

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment. Safe Robot RL (SRRL) is a crucial step towards achieving human-robot coexistence. In this paper, we envision a human-centered SRRL framework consisting of three stages: safe exploration, safety value alignment, and safe collaboration. We examine the research gaps in these areas and propose to leverage interactive behaviors for SRRL. Interactive behaviors enable bi-directional information transfer between humans and robots, such as conversational robot ChatGPT. We argue that interactive behaviors need further attention from the SRRL community. We discuss four open challenges related to the robustness, efficiency, transparency, and adaptability of SRRL with interactive behaviors.