mlsanityData to 14 Oct 2026

Continual World: A Robotic Benchmark For Continual Reinforcement Learning

arXiv 2105.10919 ↗First posted 23 May 2021cs.LG cs.AI cs.ROcited by 116

Not on the map. Why?

Cited by 35 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Continual learning (CL) -- the ability to continuously learn, building on previously acquired knowledge -- is a natural requirement for long-lived autonomous reinforcement learning (RL) agents. While building such agents, one needs to balance opposing desiderata, such as constraints on capacity and compute, the ability to not catastrophically forget, and to exhibit positive transfer on new tasks. Understanding the right trade-off is conceptually and computationally challenging, which we argue has led the community to overly focus on catastrophic forgetting. In response to these issues, we advocate for the need to prioritize forward transfer and propose Continual World, a benchmark consisting of realistic and meaningfully diverse robotic tasks built on top of Meta-World as a testbed. Following an in-depth empirical evaluation of existing CL methods, we pinpoint their limitations and highlight unique algorithmic challenges in the RL setting. Our benchmark aims to provide a meaningful and computationally inexpensive challenge for the community and thus help better understand the performance of existing and future solutions. Information about the benchmark, including the open-source code, is available at https://sites.google.com/view/continualworld.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). Builds on 39 of its 65 references that are in the map. First posted on arXiv 23 May 2021.

Builds on

39 of 65 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2016Layer Normalizationcited by 5,669
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  10. 2016Progressive Neural Networkscited by 1,361
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  12. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
27 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  2. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  3. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  5. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  6. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  7. 2017Variational Continual Learningcited by 339
  8. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 289
  9. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  10. 2019Meta-Learning Representations for Continual Learningcited by 189
  11. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 160
  12. 2018Towards Robust Evaluations of Continual Learningcited by 148
  13. 2019Continual Unsupervised Representation Learningcited by 147
  14. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  15. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 115
  16. 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
  17. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 100
  18. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  19. 2017On Quadratic Penalties in Elastic Weight Consolidationcited by 55
  20. 2019Ray Interference: a Source of Plateaus in Deep Reinforcement Learningcited by 40
  21. 2019Policy Consolidation for Continual Reinforcement Learningcited by 39
  22. 2019Continual Learning with Adaptive Weights (CLAW)cited by 38
  23. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 32
  24. 2020Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgettingcited by 29
  25. 2019Continual Reinforcement Learning in 3D Non-stationary Environmentscited by 22
  26. 2020Jelly Bean World: A Testbed for Never-Ending Learningcited by 19
  27. 2019Adaptive Online Planning for Continual Lifelong Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 14; 2023: 20; 2024: 25; 2025: 17; 2026: 36.

116 papers.

Sort:

By area:

  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  3. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 63
  4. 2023Continual Learning: Applications and the Road Forwardcited by 59
  5. 2021Powerpropagation: A sparsity inducing weight reparameterisationcited by 49
  6. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
  7. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  8. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 33
  9. 2023Avalanche: A PyTorch Library for Deep Continual Learningcited by 33
  10. 2022vCLIMB: A Novel Video Class Incremental Learning Benchmarkcited by 28
  11. 2022Building a Subspace of Policies for Scalable Continual Learningcited by 28
  12. 2023Learning to Modulate pre-trained Models in RLcited by 28
38 more
  1. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  2. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  3. 2023Low-rank extended Kalman filtering for online learning of neural networks from streaming datacited by 23
  4. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  5. 2022Lifelong Reinforcement Learning with Modulating Maskscited by 22
  6. 2024Online Continual Learning For Interactive Instruction Following Agentscited by 22
  7. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 22
  8. 2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
  9. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RLcited by 21
  10. 2024Parameter Efficient Fine-tuning via Explained Variance Adaptationcited by 20
  11. 2021Sequoia: A Software Framework to Unify Continual Learning Researchcited by 19
  12. 2022NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Researchcited by 18
  13. 2023Is forgetting less a good inductive bias for forward transfer?cited by 18
  14. 2022The Effectiveness of World Models for Continual Reinforcement Learningcited by 17
  15. 2026Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentscited by 17
  16. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  17. 2025Skill Expansion and Composition in Parameter Spacecited by 14
  18. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 13
  19. 2025Continual Learning for Generative AI: From LLMs to MLLMs and Beyondcited by 12
  20. 2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 11
  21. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  22. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 11
  23. 2023Continual Task Allocation in Meta-Policy Network via Sparse Promptingcited by 11
  24. 2024t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Makingcited by 11
  25. 2025Self-Composing Policies for Scalable Continual Reinforcement Learningcited by 11
  26. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
  27. 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Maskscited by 10
  28. 2022Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learningcited by 9
  29. 2024Continual Learning for Smart City: A Surveycited by 9
  30. 2023Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Promptcited by 8
  31. 2024A Survey Analyzing Generalization in Deep Reinforcement Learningcited by 8
  32. 2025Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learningcited by 8
  33. 2025Continual Reinforcement Learning by Planning with Online World Modelscited by 8
  34. 2023Replay-enhanced Continual Reinforcement Learningcited by 7
  35. 2024Can we hop in general? A discussion of benchmark selection and design using the Hopper environmentcited by 7
  36. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 7
  37. 2026CRL-VLA: Continual Vision-Language-Action Learningcited by 7
  38. 2026FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoffcited by 7

50 most cited of 116 All 116 (JSON)