mlsanityData to 7 Oct 2026

A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviors

arXiv 2302.13137 ↗First posted 25 Feb 2023cs.RO cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment. Safe Robot RL (SRRL) is a crucial step towards achieving human-robot coexistence. In this paper, we envision a human-centered SRRL framework consisting of three stages: safe exploration, safety value alignment, and safe collaboration. We examine the research gaps in these areas and propose to leverage interactive behaviors for SRRL. Interactive behaviors enable bi-directional information transfer between humans and robots, such as conversational robot ChatGPT. We argue that interactive behaviors need further attention from the SRRL community. We discuss four open challenges related to the robustness, efficiency, transparency, and adaptability of SRRL with interactive behaviors.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 44 of its 90 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

44 of 90 references.

References · 44

  1. 2020Learning to summarize from human feedbackcited by 2,562
  2. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  5. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  6. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  7. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  10. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  11. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  12. 2018Supervising strong learners by amplifying weak expertscited by 139
32 more
  1. 2021Collaborating with Humans without Human Datacited by 139
  2. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
  3. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  4. 2020Conservative Safety Critics for Explorationcited by 107
  5. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  6. 2018Stagewise Safe Bayesian Optimization with Gaussian Processescited by 79
  7. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  8. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 72
  9. 2019Safe Exploration for Interactive Machine Learningcited by 51
  10. 2018Contrastive Explanations for Reinforcement Learning in terms of Expected Consequencescited by 44
  11. 2019DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systemscited by 44
  12. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  13. 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
  14. 2016Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processescited by 38
  15. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  16. 2014Safe Exploration of State and Action Spaces in Reinforcement Learningcited by 37
  17. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 32
  18. 2021Multi-Agent Constrained Policy Optimisationcited by 32
  19. 2021GoSafe: Globally Optimal Safe Robot Learningcited by 27
  20. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  21. 2019Robust Regression for Safe Exploration in Controlcited by 24
  22. 2022Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopescited by 23
  23. 2019Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functionscited by 20
  24. 2021Adversarial Training is Not Ready for Robot Learningcited by 20
  25. 2021Human-robot collaboration and machine learning: a systematic review of recent researchcited by 20
  26. 2016Learning Dynamic Robot-to-Human Object Handover from Human Feedbackcited by 19
  27. 2022Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificatecited by 16
  28. 2020Robot Learning with Crash Constraintscited by 15
  29. 2020Human-centered collaborative robots with deep reinforcement learningcited by 12
  30. 2022SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulationcited by 10
  31. 2021Towards Safe, Explainable, and Regulated Autonomous Drivingcited by 7
  32. 2019Parenting: Safe Reinforcement Learning from Human Inputcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 1; 2026: 4.

13 papers.

Sort:

By area:

  1. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  2. 2024Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulationcited by 21
  3. 2024Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learningcited by 9
  4. 2024Safe Multi-Agent Reinforcement Learning with Bilevel Optimization in Autonomous Drivingcited by 9
  5. 2024Mutual Enhancement of Large Language and Reinforcement Learning Models through Bi-Directional Feedback Mechanisms: A Planning Case Studycited by 3
  6. 2024Reinforcement Learning and Machine ethics:a systematic reviewcited by 3
  7. 2025Adaptive Replay Buffer for Offline-to-Online Reinforcement Learningcited by 3
  8. 2023Machine Learning Meets Advanced Robotic Manipulationcited by 2
  9. 2026Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steeringcited by 1
  10. 2023SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimizationcited by 0
  11. 2026HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loadscited by 0
  12. 2026Learning Safe Agent Behaviour from Human Preferences and Justifications via World Modelscited by 0
1 more
  1. 2026Safe Greenhouse Climate Control Using Lagrangian-Constrained PPO with Kolmogorov-Arnold Networkscited by 0