# Initial Classifier Weights Replay for Memoryless Class Incremental Learning

arXiv: https://arxiv.org/abs/2008.13710 · HTML: https://mlsanity.com/paper/2008.13710/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Class-Incremental Learning with Generative Classifiers](https://mlsanity.com/paper/2104.10093/) (2021)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Class-Incremental Learning for Wireless Device Identification in IoT](https://mlsanity.com/paper/2105.06381/) (2021)
- [Coarse-To-Fine Incremental Few-Shot Learning](https://mlsanity.com/paper/2111.14806/) (2021)
- [Dataset Knowledge Transfer for Class-Incremental Learning without Memory](https://mlsanity.com/paper/2110.08421/) (2021)
- [PlaStIL: Plastic and Stable Memory-Free Class-Incremental Learning](https://mlsanity.com/paper/2209.06606/) (2022)
- [Coresets from Trajectories: Selecting Data via Correlation of Loss Differences](https://mlsanity.com/paper/2508.20230/) (2025)
- [Finding the Muses: Identifying Coresets through Loss Trajectories](https://mlsanity.com/paper/2503.09721/) (2025)
- [Self-distilled Knowledge Delegator for Exemplar-free Class Incremental Learning](https://mlsanity.com/paper/2205.11071/) (2022)
- [Fast Evaluation of DNN for Past Dataset in Incremental Learning](https://mlsanity.com/paper/2405.06296/) (2024)

## Abstract (arXiv)

Incremental Learning (IL) is useful when artificial systems need to deal with streams of data and do not have access to all data at all times. The most challenging setting requires a constant complexity of the deep model and an incremental model update without access to a bounded memory of past data. Then, the representations of past classes are strongly affected by catastrophic forgetting. To mitigate its negative effect, an adapted fine tuning which includes knowledge distillation is usually deployed. We propose a different approach based on a vanilla fine tuning backbone. It leverages initial classifier weights which provide a strong representation of past classes because they are trained with all class data. However, the magnitude of classifiers learned in different states varies and normalization is needed for a fair handling of all classes. Normalization is performed by standardizing the initial classifier weights, which are assumed to be normally distributed. In addition, a calibration of prediction scores is done by using state level statistics to further improve classification fairness. We conduct a thorough evaluation with four public datasets in a memoryless incremental learning setting. Results show that our method outperforms existing techniques by a large margin for large-scale datasets.