mlsanityData to 7 Oct 2026

Sequoia: A Software Framework to Unify Continual Learning Research

arXiv 2108.01005 ↗First posted 2 Aug 2021cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithmic solutions (methods) exist, each with their own potentially disjoint set of assumptions. This variety makes measuring progress in CL difficult. We propose a taxonomy of settings, where each setting is described as a set of assumptions. A tree-shaped hierarchy emerges from this view, where more general settings become the parents of those with more restrictive assumptions. This makes it possible to use inheritance to share and reuse research, as developing a method for a given setting also makes it directly applicable onto any of its children. We instantiate this idea as a publicly available software framework called Sequoia, which features a wide variety of settings from both the Continual Supervised Learning (CSL) and Continual Reinforcement Learning (CRL) domains. Sequoia also includes a growing suite of methods which are easy to extend and customize, in addition to more specialized methods from external libraries. We hope that this new paradigm and its first implementation can help unify and accelerate research in CL. You can help us grow the tree by visiting www.github.com/lebrice/Sequoia.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 52 of its 80 references that are in the map. First posted on arXiv 2 Aug 2021.

Builds on

52 of 80 references.

References · 52

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  9. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  10. 2016Progressive Neural Networkscited by 1,332
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
38 more
  1. 2017Continual Learning with Deep Generative Replaycited by 1,152
  2. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  3. 2018Experience Replay for Continual Learningcited by 1,074
  4. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  5. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  6. 2017Deep Reinforcement Learning that Matterscited by 727
  7. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  8. 2019Three scenarios for continual learningcited by 610
  9. 2019Gradient based sample selection for online continual learningcited by 568
  10. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  11. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  12. 2017Variational Continual Learningcited by 335
  13. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
  14. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  15. 2018Task-Free Continual Learningcited by 219
  16. 2015The Benefit of Multitask Representation Learningcited by 194
  17. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  18. 2018Towards Robust Evaluations of Continual Learningcited by 148
  19. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 133
  20. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  21. 2018Generalization and Regularization in DQNcited by 117
  22. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  23. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  24. 2020Efficient Continual Learning with Modular Networks and Task-Driven Priorscited by 87
  25. 2018Generative Models from the perspective of Continual Learningcited by 69
  26. 2018Task Agnostic Continual Learning Using Online Variational Bayescited by 61
  27. 2019Task Agnostic Continual Learning via Meta Learningcited by 53
  28. 2020La-MAML: Look-ahead Meta Learning for Continual Learningcited by 51
  29. 2019Continuous Meta-Learning without Taskscited by 50
  30. 2020Deep Reinforcement Learning amidst Lifelong Non-Stationaritycited by 42
  31. 2020Optimizing for the Future in Non-Stationary MDPscited by 37
  32. 2019DisCoRL: Continual Reinforcement Learning via Policy Distillationcited by 35
  33. 2021Continuum: Simple Management of Complex Continual Learning Scenarioscited by 33
  34. 2020Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgettingcited by 29
  35. 2019Continual Learning Using Bayesian Neural Networkscited by 19
  36. 2019Lifelong Learning with a Changing Action Setcited by 17
  37. 2019A Model-based Approach for Sample-efficient Multi-task Reinforcement Learningcited by 12
  38. 2018Marginal Replay vs Conditional Replay for Continual Learningcited by 10

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 5; 2024: 3; 2025: 1; 2026: 1.

19 papers.

Sort:

By area:

  1. 2021Continual Learning via Local Module Compositioncited by 61
  2. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  3. 2023Avalanche: A PyTorch Library for Deep Continual Learningcited by 33
  4. 2024Position: Theory of Mind Benchmarks are Broken for Large Language Modelscited by 30
  5. 2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
  6. 2022Continual Object Detection: A review of definitions, strategies, and challengescited by 25
  7. 2023A Comprehensive Empirical Evaluation on Online Continual Learningcited by 24
  8. 2022Dynamic Dialogue Policy for Continual Reinforcement Learningcited by 12
  9. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  10. 2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 10
  11. 2024Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inferencecited by 7
  12. 2023Efficient Parametric Approximations of Neural Network Function Space Distancecited by 6
7 more
  1. 2022Avalanche RL: a Continual Reinforcement Learning Librarycited by 4
  2. 2022A Study of Continual Learning Methods for Q-Learningcited by 3
  3. 2023Renate: A Library for Real-World Continual Learningcited by 2
  4. 2025The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Modelscited by 2
  5. 2023SequeL: A Continual Learning Library in PyTorch and JAXcited by 1
  6. 2026Position: Collusion Risks Among AI Reasoning Agents Justify Certification Requirements for Making Market Decisionscited by 1
  7. 2024Deep evolving semi-supervised anomaly detectioncited by 0