mlsanityData to 14 Oct 2026

A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks

arXiv 2506.02883 ↗First posted 3 Jun 2025cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Autonomous agents operating in domains such as robotics or video game simulations must adapt to changing tasks without forgetting about the previous ones. This process called Continual Reinforcement Learning poses non-trivial difficulties, from preventing catastrophic forgetting to ensuring the scalability of the approaches considered. Building on recent advances, we introduce a benchmark providing a suite of video-game navigation scenarios, thus filling a gap in the literature and capturing key challenges : catastrophic forgetting, task adaptation, and memory efficiency. We define a set of various tasks and datasets, evaluation protocols, and metrics to assess the performance of algorithms, including state-of-the-art baselines. Our benchmark is designed not only to foster reproducible research and to accelerate progress in continual reinforcement learning for gaming, but also to provide a reproducible framework for production pipelines -- helping practitioners to identify and to apply effective approaches.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 34 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

29 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning19 Dec
  2. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay16 Apr
  3. 2023Learning Computational Efficient Bots with Costly Features18 Aug

Other references · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Layer Normalizationcited by 5,669
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2016Progressive Neural Networkscited by 1,361
  6. 2018Experience Replay for Continual Learningcited by 1,095
  7. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  8. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  10. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  11. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  12. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
14 more
  1. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  2. 2019Goal-conditioned Imitation Learningcited by 162
  3. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  4. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  5. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 115
  6. 2021Learning Neural Network Subspacescited by 82
  7. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  8. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  9. 2020Continual Model-Based Reinforcement Learning with Hypernetworkscited by 44
  10. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
  11. 2017The Atari Grand Challenge Datasetcited by 14
  12. 2024Efficient Active Imitation Learning with Random Network Distillationcited by 7
  13. 2020Towards Game-Playing AI Benchmarks via Performance Reporting Standardscited by 3
  14. 2024A Benchmark Environment for Offline Reinforcement Learning in Racing Gamescited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025A Survey of Continual Reinforcement Learningcited by 18
  2. 2025AbideGym: Turning Static RL Worlds into Adaptive Challengescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  2. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning
  3. 2025Rethinking the Foundations for Continual Reinforcement Learning
  4. 2022The Effectiveness of World Models for Continual Reinforcement Learning
  5. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning
  6. 2025Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments
  7. 2025Continual Reinforcement Learning by Planning with Online World Models
  8. 2025An Empirical Study of Deep Reinforcement Learning in Continuing Tasks