# A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

arXiv: https://arxiv.org/abs/2005.10619 · HTML: https://mlsanity.com/paper/2005.10619/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2020.

## Built on by (most cited first, 30 of 62)

- [Universal Off-Policy Evaluation](https://mlsanity.com/paper/2104.12820/) (2021)
- [Continual Learning as Computationally Constrained Reinforcement Learning](https://mlsanity.com/paper/2307.04345/) (2023)
- [Factored Adaptation for Non-Stationary Reinforcement Learning](https://mlsanity.com/paper/2203.16582/) (2022)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Neurosymbolic Reinforcement Learning and Planning: A Survey](https://mlsanity.com/paper/2309.01038/) (2023)
- [Minimum-Delay Adaptation in Non-Stationary Reinforcement Learning via Online High-Confidence Change-Point Detection](https://mlsanity.com/paper/2105.09452/) (2021)
- [RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN](https://mlsanity.com/paper/2111.06978/) (2021)
- [The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models](https://mlsanity.com/paper/2402.01874/) (2024)
- [Manipulating Recommender Systems: A Survey of Poisoning Attacks and Countermeasures](https://mlsanity.com/paper/2404.14942/) (2024)
- [Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation](https://mlsanity.com/paper/2210.03209/) (2022)
- [Beyond Supervised Continual Learning: a Review](https://mlsanity.com/paper/2208.14307/) (2022)
- [Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective](https://mlsanity.com/paper/2501.03562/) (2025)
- [Python-Based Reinforcement Learning on Simulink Models](https://mlsanity.com/paper/2405.08567/) (2024)
- [SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines](https://mlsanity.com/paper/2408.04323/) (2024)
- [Meta-Reinforcement Learning by Tracking Task Non-stationarity](https://mlsanity.com/paper/2105.08834/) (2021)
- [The complexity of non-stationary reinforcement learning](https://mlsanity.com/paper/2307.06877/) (2023)
- [Dealing with Non-Stationarity in MARL via Trust-Region Decomposition](https://mlsanity.com/paper/2102.10616/) (2021)
- [Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization](https://mlsanity.com/paper/2112.06625/) (2021)
- [Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representation](https://mlsanity.com/paper/2306.02747/) (2023)
- [DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs](https://mlsanity.com/paper/2601.14711/) (2026)
- [Online reinforcement learning with sparse rewards through an active inference capsule](https://mlsanity.com/paper/2106.02390/) (2021)
- [System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games](https://mlsanity.com/paper/2212.04603/) (2022)
- [Tempo Adaptation in Non-stationary Reinforcement Learning](https://mlsanity.com/paper/2309.14989/) (2023)
- [Offline Reinforcement Learning with Domain-Unlabeled Data](https://mlsanity.com/paper/2404.07465/) (2024)
- [Sample Efficient Experience Replay in Non-stationary Environments](https://mlsanity.com/paper/2509.15032/) (2025)
- [GLOVE: Global Verifier for LLM Memory-Environment Realignment](https://mlsanity.com/paper/2601.19249/) (2026)
- [Collaboration Promotes Group Resilience in Multi-Agent RL](https://mlsanity.com/paper/2111.06614/) (2021)
- [Off-Policy Evaluation for Action-Dependent Non-Stationary Environments](https://mlsanity.com/paper/2301.10330/) (2023)
- [Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures](https://mlsanity.com/paper/2305.18701/) (2023)
- [Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications](https://mlsanity.com/paper/2309.10177/) (2023)

## Abstract (arXiv)

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing and robotics. The real-world complications of many tasks arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This paper provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent which leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally we also review works which are tailored to application domains. Finally, we discuss future enhancements for this field.