mlsanityData to 14 Oct 2026

A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points

arXiv 2405.14214 ↗First posted 23 May 2024cs.LG cs.AIcited by 5

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Deep reinforcement learning is used in various domains, but usually under the assumption that the environment has stationary conditions like transitions and state distributions. When this assumption is not met, performance suffers. For this reason, tracking continuous environmental changes and adapting to unpredictable conditions is challenging yet crucial because it ensures that systems remain reliable and flexible in practical scenarios. Our research introduces Behavior-Aware Detection and Adaptation (BADA), an innovative framework that merges environmental change detection with behavior adaptation. The key inspiration behind our method is that policies exhibit different global behaviors in changing environments. Specifically, environmental changes are identified by analyzing variations between behaviors using Wasserstein distances without manually set thresholds. The model adapts to the new environment through behavior regularization based on the extent of changes. The results of a series of experiments demonstrate better performance relative to several current algorithms. This research also indicates significant potential for tackling this long-standing challenge.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 43 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

22 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learning17 Jan

Other references · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017Continual Learning with Deep Generative Replaycited by 1,158
  7. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  8. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  9. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  10. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  11. 2018Statistical Aspects of Wasserstein Distancescited by 203
  12. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
9 more
  1. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  2. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 62
  3. 2020Reinforcement Learning with Combinatorial Actions: An Application to Vehicle Routingcited by 61
  4. 2018ViZDoom Competitions: Playing Doom from Pixelscited by 51
  5. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
  6. 2019Learning to Score Behaviors for Guided Policy Optimizationcited by 34
  7. 2022AnyMorph: Learning Transferable Polices By Inferring Agent Morphologycited by 27
  8. 2020Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economicscited by 23
  9. 2022Reinforcement Learning in Medical Image Analysis: Concepts, Applications, Challenges, and Future Directionscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measurescited by 0
  2. 2025Wavelet Predictive Representations for Non-Stationary Reinforcement Learningcited by 0
  3. 2026A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learningcited by 0
  4. 2026Changepoint-Aware World Models: Detecting Dynamics Shifts and Recovering by Forgetting Stale Replay in Model-Based RLcited by 0
  5. 2026Diagnose, Recover, Certify: Task Readiness under Hidden Dynamics Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Adaptation of Reinforcement Learning Agents to Sudden Environmental Change
  2. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  3. 2022A Behavior Regularized Implicit Policy for Offline Reinforcement Learning
  4. 2025Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning
  5. 2025Self-Consistent Model-based Adaptation for Visual Reinforcement Learning
  6. 2025Behavioral Exploration: Learning to Explore via In-Context Adaptation
  7. 2025Sample Efficient Experience Replay in Non-stationary Environments
  8. 2024Partial Models for Building Adaptive Model-Based Reinforcement Learning Agents