mlsanityData to 14 Oct 2026

Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measures

arXiv 2509.11367 ↗First posted 14 Sep 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) agents typically assume stationary environment dynamics. Yet in real-world applications such as healthcare, robotics, and finance, transition probabilities or reward functions may evolve, leading to model drift. This paper proposes a novel framework to detect such drifts by analyzing the distributional changes in sequences of agent behavior. Specifically, we introduce a suite of edit operation-based measures to quantify deviations between state-action trajectories generated under stationary and perturbed conditions. Our experiments demonstrate that these measures can effectively distinguish drifted from non-drifted scenarios, even under varying levels of noise, providing a practical tool for drift detection in non-stationary RL environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 24 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

9 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points23 May
  2. 2024Assessing the Impact of Distribution Shift on Reinforcement Learning Performance5 Feb

Other references · 7

  1. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  2. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  3. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
  4. 2022Testing Stationarity and Change Point Detection in Reinforcement Learningcited by 12
  5. 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
  6. 2024Sample Complexity Characterization for Linear Contextual MDPscited by 5
  7. 2024Offline Reinforcement Learning from Datasets with Structured Non-Stationaritycited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Novelty Detection in Reinforcement Learning with World Models
  2. 2026A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning
  3. 2025Forecasting in Offline Reinforcement Learning for Non-stationary Environments
  4. 2025Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies
  5. 2024Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection
  6. 2026OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning
  7. 2025Model-Based Reinforcement Learning under Random Observation Delays
  8. 2024Mitigating Distribution Shift in Model-based Offline RL via Shifts-aware Reward Learning