# Multi-Domain Multi-Task Rehearsal for Lifelong Learning

arXiv: https://arxiv.org/abs/2012.07236 · HTML: https://mlsanity.com/paper/2012.07236/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 24 of 24)

- [Exploring Example Influence in Continual Learning](https://mlsanity.com/paper/2209.12241/) (2022)
- [AGCN: Augmented Graph Convolutional Network for Lifelong Multi-label Image Recognition](https://mlsanity.com/paper/2203.05534/) (2022)
- [Multi-Label Continual Learning using Augmented Graph Convolutional Network](https://mlsanity.com/paper/2211.14763/) (2022)
- [Disentangling Semantic-to-visual Confusion for Zero-shot Learning](https://mlsanity.com/paper/2106.08605/) (2021)
- [Variational Continual Test-Time Adaptation](https://mlsanity.com/paper/2402.08182/) (2024)
- [Towards Continuous Reuse of Graph Models via Holistic Memory Diversification](https://mlsanity.com/paper/2406.07413/) (2024)
- [Confidence Self-Calibration for Multi-Label Class-Incremental Learning](https://mlsanity.com/paper/2403.12559/) (2024)
- [Controllable Continual Test-Time Adaptation](https://mlsanity.com/paper/2405.14602/) (2024)
- [Parameter-Selective Continual Test-Time Adaptation](https://mlsanity.com/paper/2407.02253/) (2024)
- [MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation](https://mlsanity.com/paper/2506.14436/) (2025)
- [Centroid Distance Distillation for Effective Rehearsal in Continual Learning](https://mlsanity.com/paper/2303.02954/) (2023)
- [Long-Tailed Learning as Multi-Objective Optimization](https://mlsanity.com/paper/2310.20490/) (2023)
- [Constructing Sample-to-Class Graph for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2310.20268/) (2023)
- [Elastic Multi-Gradient Descent for Parallel Continual Learning](https://mlsanity.com/paper/2401.01054/) (2024)
- [Overcoming Domain Drift in Online Continual Learning](https://mlsanity.com/paper/2405.09133/) (2024)
- [Improving Data-aware and Parameter-aware Robustness for Continual Learning](https://mlsanity.com/paper/2405.17054/) (2024)
- [Rebalancing Multi-Label Class-Incremental Learning](https://mlsanity.com/paper/2408.12161/) (2024)
- [Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing](https://mlsanity.com/paper/2110.11159/) (2021)
- [Always Strengthen Your Strengths: A Drift-Aware Incremental Learning Framework for CTR Prediction](https://mlsanity.com/paper/2304.09062/) (2023)
- [Recall-Oriented Continual Learning with Generative Adversarial Meta-Model](https://mlsanity.com/paper/2403.03082/) (2024)
- [MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering](https://mlsanity.com/paper/2505.19455/) (2025)
- [Two-level Graph Network for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2303.13862/) (2023)
- [Towards stable training of parallel continual learning](https://mlsanity.com/paper/2407.08214/) (2024)
- [Ambiguity-Guided Learnable Distribution Calibration for Semi-Supervised Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2507.23237/) (2025)

## Abstract (arXiv)

Rehearsal, seeking to remind the model by storing old knowledge in lifelong learning, is one of the most effective ways to mitigate catastrophic forgetting, i.e., biased forgetting of previous knowledge when moving to new tasks. However, the old tasks of the most previous rehearsal-based methods suffer from the unpredictable domain shift when training the new task. This is because these methods always ignore two significant factors. First, the Data Imbalance between the new task and old tasks that makes the domain of old tasks prone to shift. Second, the Task Isolation among all tasks will make the domain shift toward unpredictable directions; To address the unpredictable domain shift, in this paper, we propose Multi-Domain Multi-Task (MDMT) rehearsal to train the old tasks and new task parallelly and equally to break the isolation among tasks. Specifically, a two-level angular margin loss is proposed to encourage the intra-class/task compactness and inter-class/task discrepancy, which keeps the model from domain chaos. In addition, to further address domain shift of the old tasks, we propose an optional episodic distillation loss on the memory to anchor the knowledge for each old task. Experiments on benchmark datasets validate the proposed approach can effectively mitigate the unpredictable domain shift.