# Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments

arXiv: https://arxiv.org/abs/2609.11433 · HTML: https://mlsanity.com/paper/2609.11433/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation](https://mlsanity.com/paper/2008.07682/) (2020)
- [Learning Barrier Functions with Memory for Robust Safe Navigation](https://mlsanity.com/paper/2011.01899/) (2020)
- [Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction](https://mlsanity.com/paper/2209.13308/) (2022)
- [Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models](https://mlsanity.com/paper/2111.13129/) (2021)
- [Multi-query Robotic Manipulator Task Sequencing with Gromov-Hausdorff Approximations](https://mlsanity.com/paper/2209.04800/) (2022)
- [Interactive Distance Field Mapping and Planning to Enable Human-Robot Collaboration](https://mlsanity.com/paper/2403.09988/) (2024)
- [A Reactive Framework for Whole-Body Motion Planning of Mobile Manipulators Combining Reinforcement Learning and SDF-Constrained Quadratic Programmi](https://mlsanity.com/paper/2503.23975/) (2025)
- [Towards Robot Skill Learning and Adaptation with Gaussian Processes](https://mlsanity.com/paper/2603.01480/) (2026)

## Abstract (arXiv)

Skill adaptation frameworks based on reinforcement learning often require restrictive assumptions to maintain stability, such as fixed observations or tightly controlled exploration schedules. In cluttered and dynamic environments, however, unrestricted exploration can lead to unsafe behaviour and unstable learning, particularly when task-relevant observations lie near obstacles or involve moving objects. In this work, we present Dist-GPRL, a distance-aware and safety-guided reinforcement learning framework for structured robot skill adaptation. Building upon Gaussian Process (GP)-based skill parameterisation, our framework sequentially adapts overlapping local windows of sparse trajectory via-points rather than modifying the complete skill at every policy step. Raw policy outputs are correlated through the GP covariance structure, producing temporally coherent trajectory updates while reducing the action-space and credit-assignment difficulties associated with global trajectory adaptation. Safety is incorporated through two complementary forms of guidance. A safe-subspace prior derived from the Hausdorff Approximation Planner (HAP) biases policy exploration toward feasible regions, while dynamically updated distance field clearance and gradient rewards provide local obstacle awareness. A trajectory-kinematics similarity regulariser further preserves the demonstrated velocity and acceleration characteristics during adaptation. We evaluate the framework on two dynamic object-manipulation tasks in simulation and transfer the learned policy to real-world robot execution. Experimental results demonstrate higher task success, lower collision frequency, and more stable learning than the baselines, while preserving the kinematic characteristics of the demonstrated skill.