# SequeL: A Continual Learning Library in PyTorch and JAX

arXiv: https://arxiv.org/abs/2304.10857 · HTML: https://mlsanity.com/paper/2304.10857/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [FractalNet: Ultra-Deep Neural Networks without Residuals](https://mlsanity.com/paper/1605.07648/) (2016)
- [fastai: A Layered API for Deep Learning](https://mlsanity.com/paper/2002.04688/) (2020)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Linear Mode Connectivity in Multitask and Continual Learning](https://mlsanity.com/paper/2010.04495/) (2020)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Continuum: Simple Management of Complex Continual Learning Scenarios](https://mlsanity.com/paper/2102.06253/) (2021)
- [Accelerator-aware Neural Network Design using AutoML](https://mlsanity.com/paper/2003.02838/) (2020)
- [Less-forgetful Learning for Domain Expansion in Deep Neural Networks](https://mlsanity.com/paper/1711.05959/) (2017)
- [Kernel Continual Learning](https://mlsanity.com/paper/2107.05757/) (2021)
- [Sequoia: A Software Framework to Unify Continual Learning Research](https://mlsanity.com/paper/2108.01005/) (2021)
- [Avalanche RL: a Continual Reinforcement Learning Library](https://mlsanity.com/paper/2202.13657/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Empirical Evaluation on Online Continual Learning](https://mlsanity.com/paper/2308.10328/) (2023)

## Abstract (arXiv)

Continual Learning is an important and challenging problem in machine learning, where models must adapt to a continuous stream of new data without forgetting previously acquired knowledge. While existing frameworks are built on PyTorch, the rising popularity of JAX might lead to divergent codebases, ultimately hindering reproducibility and progress. To address this problem, we introduce SequeL, a flexible and extensible library for Continual Learning that supports both PyTorch and JAX frameworks. SequeL provides a unified interface for a wide range of Continual Learning algorithms, including regularization-based approaches, replay-based approaches, and hybrid approaches. The library is designed towards modularity and simplicity, making the API suitable for both researchers and practitioners. We release SequeLfootnoteurlhttps://github.com/nik-dim/sequel as an open-source library, enabling researchers and developers to easily experiment and extend the library for their own purposes.