# Continual Sequence Generation with Adaptive Compositional Modules

arXiv: https://arxiv.org/abs/2203.10652 · HTML: https://mlsanity.com/paper/2203.10652/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 56 references that are in the map.
- First posted on arXiv 20 Mar 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [AdapterDrop: On the Efficiency of Adapters in Transformers](https://mlsanity.com/paper/2010.11918/) (2020)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning](https://mlsanity.com/paper/2106.09701/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Continual Learning for Text Classification with Information Disentanglement Based Regularization](https://mlsanity.com/paper/2104.05489/) (2021)
- [Continual Learning in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2012.15504/) (2020)
- [Learning and Evaluating General Linguistic Intelligence](https://mlsanity.com/paper/1901.11373/) (2019)

## Built on by (most cited first, 30 of 31)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Mitigating the Alignment Tax of RLHF](https://mlsanity.com/paper/2309.06256/) (2023)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)
- [CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks](https://mlsanity.com/paper/2206.09059/) (2022)
- [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024)
- [InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions](https://mlsanity.com/paper/2403.11435/) (2024)
- [Investigating Continual Pretraining in Large Language Models: Insights and Implications](https://mlsanity.com/paper/2402.17400/) (2024)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models](https://mlsanity.com/paper/2401.08295/) (2024)
- [Rehearsal-Free Modular and Compositional Continual Learning for Language Models](https://mlsanity.com/paper/2404.00790/) (2024)
- [Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation](https://mlsanity.com/paper/2504.00420/) (2025)
- [UDApter -- Efficient Domain Adaptation Using Adapters](https://mlsanity.com/paper/2302.03194/) (2023)
- [Class-Incremental Learning based on Label Generation](https://mlsanity.com/paper/2306.12619/) (2023)
- [Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation](https://mlsanity.com/paper/2310.09886/) (2023)
- [Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering](https://mlsanity.com/paper/2502.04469/) (2025)
- [I2I: Initializing Adapters with Improvised Knowledge](https://mlsanity.com/paper/2304.02168/) (2023)
- [Continual Learning with Dirichlet Generative-based Rehearsal](https://mlsanity.com/paper/2309.06917/) (2023)
- [Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks](https://mlsanity.com/paper/2310.09436/) (2023)
- [Self-generated Replay Memories for Continual Neural Machine Translation](https://mlsanity.com/paper/2403.13130/) (2024)
- [Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Models](https://mlsanity.com/paper/2404.10306/) (2024)
- [Learn it or Leave it: Module Composition and Pruning for Continual Learning](https://mlsanity.com/paper/2406.18708/) (2024)
- [HiCL: Hippocampal-Inspired Continual Learning](https://mlsanity.com/paper/2508.16651/) (2025)
- [MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learning](https://mlsanity.com/paper/2603.16738/) (2026)
- [Semi-Supervised Lifelong Language Learning](https://mlsanity.com/paper/2211.13050/) (2022)
- [Towards Zero-Shot Functional Compositionality of Language Models](https://mlsanity.com/paper/2303.03103/) (2023)
- [Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis](https://mlsanity.com/paper/2303.15493/) (2023)
- [Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments](https://mlsanity.com/paper/2401.10393/) (2024)
- [Sparse Adapter Fusion for Continual Learning in NLP](https://mlsanity.com/paper/2602.02502/) (2026)
- [LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective](https://mlsanity.com/paper/2606.26183/) (2026)

## Abstract (arXiv)

Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual sequence generation either always reuses existing parameters to learn new tasks, which is vulnerable to catastrophic forgetting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between similar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowledge transfer in those shared modules. Experiment results on various sequences of generation tasks show that our framework can adaptively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and parameter efficiency. We make our code public at https://github.com/GT-SALT/Adaptive-Compositional-Modules.