# ScaIL: Classifier Weights Scaling for Class Incremental Learning

arXiv: https://arxiv.org/abs/2001.05755 · HTML: https://mlsanity.com/paper/2001.05755/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2020.

## Built on by (most cited first, 30 of 47)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Dataset Pruning: Reducing Training Data by Examining Generalization Influence](https://mlsanity.com/paper/2205.09329/) (2022)
- [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022)
- [Efficient Dataset Distillation Using Random Feature Approximation](https://mlsanity.com/paper/2210.12067/) (2022)
- [DataDAM: Efficient Dataset Distillation with Attention Matching](https://mlsanity.com/paper/2310.00093/) (2023)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [DC-BENCH: Dataset Condensation Benchmark](https://mlsanity.com/paper/2207.09639/) (2022)
- [Graph Condensation via Receptive Field Distribution Matching](https://mlsanity.com/paper/2206.13697/) (2022)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Delving into Effective Gradient Matching for Dataset Condensation](https://mlsanity.com/paper/2208.00311/) (2022)
- [Class-Incremental Learning with Generative Classifiers](https://mlsanity.com/paper/2104.10093/) (2021)
- [Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting](https://mlsanity.com/paper/2108.08165/) (2021)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2307.02246/) (2023)
- [Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2210.07207/) (2022)
- [Improving Vision Transformers for Incremental Learning](https://mlsanity.com/paper/2112.06103/) (2021)
- [Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation](https://mlsanity.com/paper/2106.15287/) (2021)
- [Neural Spectral Decomposition for Dataset Distillation](https://mlsanity.com/paper/2408.16236/) (2024)
- [Few-shot Class-incremental Learning for 3D Point Cloud Objects](https://mlsanity.com/paper/2205.15225/) (2022)
- [Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting](https://mlsanity.com/paper/2102.01906/) (2021)
- [Coarse-To-Fine Incremental Few-Shot Learning](https://mlsanity.com/paper/2111.14806/) (2021)
- [Self-Supervised Features Improve Open-World Learning](https://mlsanity.com/paper/2102.07848/) (2021)
- [PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning](https://mlsanity.com/paper/2407.14029/) (2024)
- [A First Look at Class Incremental Learning in Deep Learning Mobile Traffic Classification](https://mlsanity.com/paper/2107.04464/) (2021)
- [Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning](https://mlsanity.com/paper/2206.07842/) (2022)
- [Bayesian Pseudo-Coresets via Contrastive Divergence](https://mlsanity.com/paper/2303.11278/) (2023)
- [Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings](https://mlsanity.com/paper/2410.20768/) (2024)

## Abstract (arXiv)

Incremental learning is useful if an AI agent needs to integrate data from a stream. The problem is non trivial if the agent runs on a limited computational budget and has a bounded memory of past data. In a deep learning approach, the constant computational budget requires the use of a fixed architecture for all incremental states. The bounded memory generates data imbalance in favor of new classes and a prediction bias toward them appears. This bias is commonly countered by introducing a data balancing step in addition to the basic network training. We depart from this approach and propose simple but efficient scaling of past class classifier weights to make them more comparable to those of new classes. Scaling exploits incremental state level statistics and is applied to the classifiers learned in the initial state of classes in order to profit from all their available data. We also question the utility of the widely used distillation loss component of incremental learning algorithms by comparing it to vanilla fine tuning in presence of a bounded memory. Evaluation is done against competitive baselines using four public datasets. Results show that the classifier weights scaling and the removal of the distillation are both beneficial.