# Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review

arXiv: https://arxiv.org/abs/2506.21899 · HTML: https://mlsanity.com/paper/2506.21899/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- First posted on arXiv 27 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Built on by (most cited first, 2 of 2)

- [PATH-Bench: Path-Dependent Evaluation of Lifelong Agents](https://mlsanity.com/paper/2608.01149/) (2026)
- [Self-adapting Robotic Agents through Online Continual Reinforcement Learning with World Model Feedback](https://mlsanity.com/paper/2603.04029/) (2026)

## Abstract (arXiv)

The diversity of tasks and dynamic nature of reinforcement learning (RL) require RL agents to be able to learn sequentially and continuously, a learning paradigm known as continuous reinforcement learning. This survey reviews how continual learning transforms RL agents into dynamic continual learners. This enables RL agents to acquire and retain useful and reusable knowledge seamlessly. The paper delves into fundamental aspects of continual reinforcement learning, exploring key concepts, significant challenges, and novel methodologies. Special emphasis is placed on recent advancements in continual reinforcement learning within robotics, along with a succinct overview of evaluation environments utilized in prominent research, facilitating accessibility for newcomers to the field. The review concludes with a discussion on limitations and promising future directions, providing valuable insights for researchers and practitioners alike.