# Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning

arXiv: https://arxiv.org/abs/2212.14284 · HTML: https://mlsanity.com/paper/2212.14284/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 53 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 30 of 39)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion](https://mlsanity.com/paper/2111.11326/) (2021)
- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [A continual learning survey: Defying forgetting in classification tasks](https://mlsanity.com/paper/1909.08383/) (2019)
- [Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning](https://mlsanity.com/paper/2203.06359/) (2022)
- [Continual Learning via Neural Pruning](https://mlsanity.com/paper/1903.04476/) (2019)
- [Essentials for Class Incremental Learning](https://mlsanity.com/paper/2102.09517/) (2021)

## Built on by (most cited first, 15 of 15)

- [Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning](https://mlsanity.com/paper/2403.12030/) (2024)
- [Task-Agnostic Guided Feature Expansion for Class-Incremental Learning](https://mlsanity.com/paper/2503.00823/) (2025)
- [MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition](https://mlsanity.com/paper/2305.14758/) (2023)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [Do Your Best and Get Enough Rest for Continual Learning](https://mlsanity.com/paper/2503.18371/) (2025)
- [CroMo-Mixup: Augmenting Cross-Model Representations for Continual Self-Supervised Learning](https://mlsanity.com/paper/2407.12188/) (2024)
- [Energy-Structured Low-Rank Adaptation for Continual Learning](https://mlsanity.com/paper/2605.27482/) (2026)
- [MASIL: Towards Maximum Separable Class Representation for Few Shot Class Incremental Learning](https://mlsanity.com/paper/2304.05362/) (2023)
- [Similarity-based context aware continual learning for spiking neural networks](https://mlsanity.com/paper/2411.05802/) (2024)
- [Addressing Imbalance for Class Incremental Learning in Medical Image Classification](https://mlsanity.com/paper/2407.13768/) (2024)
- [Reducing catastrophic forgetting of incremental learning in the absence of rehearsal memory with task-specific token](https://mlsanity.com/paper/2411.05846/) (2024)
- [Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning](https://mlsanity.com/paper/2603.08426/) (2026)
- [Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning](https://mlsanity.com/paper/2603.09145/) (2026)
- [Pruned Adaptation Modules: A Simple yet Strong Baseline for Continual Foundation Models](https://mlsanity.com/paper/2603.21170/) (2026)
- [DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning](https://mlsanity.com/paper/2607.01630/) (2026)

## Abstract (arXiv)

The dynamic expansion architecture is becoming popular in class incremental learning, mainly due to its advantages in alleviating catastrophic forgetting. However, task confusion is not well assessed within this framework, e.g., the discrepancy between classes of different tasks is not well learned (i.e., inter-task confusion, ITC), and certain priority is still given to the latest class batch (i.e., old-new confusion, ONC). We empirically validate the side effects of the two types of confusion. Meanwhile, a novel solution called Task Correlated Incremental Learning (TCIL) is proposed to encourage discriminative and fair feature utilization across tasks. TCIL performs a multi-level knowledge distillation to propagate knowledge learned from old tasks to the new one. It establishes information flow paths at both feature and logit levels, enabling the learning to be aware of old classes. Besides, attention mechanism and classifier re-scoring are applied to generate more fair classification scores. We conduct extensive experiments on CIFAR100 and ImageNet100 datasets. The results demonstrate that TCIL consistently achieves state-of-the-art accuracy. It mitigates both ITC and ONC, while showing advantages in battle with catastrophic forgetting even no rehearsal memory is reserved.