mlsanityData to 14 Oct 2026

Safe Explicable Policy Search

arXiv 2503.07848 ↗First posted 10 Mar 2025cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

When users work with AI agents, they form conscious or subconscious expectations of them. Meeting user expectations is crucial for such agents to engage in successful interactions and teaming. However, users may form expectations of an agent that differ from the agent's planned behaviors. These differences lead to the consideration of two separate decision models in the planning process to generate explicable behaviors. However, little has been done to incorporate safety considerations, especially in a learning setting. We present Safe Explicable Policy Search (SEPS), which aims to provide a learning approach to explicable behavior generation while minimizing the safety risk, both during and after learning. We formulate SEPS as a constrained optimization problem where the agent aims to maximize an explicability score subject to constraints on safety and a suboptimality criterion based on the agent's model. SEPS innovatively combines the capabilities of Constrained Policy Optimization and Explicable Policy Search to introduce the capability of generating safe explicable behaviors to domains with continuous state and action spaces, which is critical for robotic applications. We evaluate SEPS in safety-gym environments and with a physical robot experiment to show its efficacy and relevance in human-AI teaming.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

25 of 46 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  6. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  7. 2017Safe Reinforcement Learning via Shieldingcited by 449
  8. 2018Reward Constrained Policy Optimizationcited by 417
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  10. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  11. 2020Projection-Based Constrained Policy Optimizationcited by 203
  12. 2017A Rewriting System for Convex Optimization Problemscited by 179
13 more
  1. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 158
  2. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  3. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  4. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  5. 2022Models of human preference for learning reward functionscited by 54
  6. 2017Explainable Planningcited by 40
  7. 2018Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behaviorcited by 40
  8. 2018Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behaviorcited by 30
  9. 2015Plan Explicability and Predictability for Robot Task Planningcited by 26
  10. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  11. 2016Explicablility as Minimizing Distance from Expected Behaviorcited by 3
  12. 2021Generating Active Explicable Plans in Human-Robot Teamingcited by 2
  13. 2023Safe Explicable Planningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Safe Reinforcement Learning in a Simulated Robotic Arm
  2. 2024Robots that Suggest Safe Alternatives
  3. 2025Safety Representations for Safer Policy Learning
  4. 2026Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning
  5. 2026OSIL: Learning Offline Safe Imitation Policies with Safety Inferred from Non-preferred Trajectories
  6. 2022SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation
  7. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning
  8. 2024A Safe Exploration Strategy for Model-free Task Adaptation in Safety-constrained Grid Environments