# Online Hyperparameter Optimization for Class-Incremental Learning

arXiv: https://arxiv.org/abs/2301.05032 · HTML: https://mlsanity.com/paper/2301.05032/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Adaptive Aggregation Networks for Class-Incremental Learning](https://mlsanity.com/paper/2010.05063/) (2020)
- [Distilling Causal Effect of Data in Class-Incremental Learning](https://mlsanity.com/paper/2103.01737/) (2021)
- [Conditional Channel Gated Networks for Task-Aware Continual Learning](https://mlsanity.com/paper/2004.00070/) (2020)
- [Encoder Based Lifelong Learning](https://mlsanity.com/paper/1704.01920/) (2017)
- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Online Optimal Control with Affine Constraints](https://mlsanity.com/paper/2010.04891/) (2020)

## Built on by (most cited first, 30 of 36)

- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Continual Detection Transformer for Incremental Object Detection](https://mlsanity.com/paper/2304.03110/) (2023)
- [Class-Incremental Exemplar Compression for Class-Incremental Learning](https://mlsanity.com/paper/2303.14042/) (2023)
- [Continual Learning for Abdominal Multi-Organ and Tumor Segmentation](https://mlsanity.com/paper/2306.00988/) (2023)
- [Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learning](https://mlsanity.com/paper/2508.08165/) (2025)
- [Generating Images with 3D Annotations Using Diffusion Models](https://mlsanity.com/paper/2306.08103/) (2023)
- [Heterogeneous Forgetting Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2308.03374/) (2023)
- [Wakening Past Concepts without Past Data: Class-Incremental Learning from Online Placebos](https://mlsanity.com/paper/2310.16115/) (2023)
- [Hyperparameters in Continual Learning: A Reality Check](https://mlsanity.com/paper/2403.09066/) (2024)
- [How to Teach Large Multimodal Models New Skills](https://mlsanity.com/paper/2510.08564/) (2025)
- [iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning](https://mlsanity.com/paper/2407.09271/) (2024)
- [REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning](https://mlsanity.com/paper/2403.13522/) (2024)
- [Continual Adversarial Defense](https://mlsanity.com/paper/2312.09481/) (2023)
- [Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task](https://mlsanity.com/paper/2405.17779/) (2024)
- [AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data](https://mlsanity.com/paper/2505.12245/) (2025)
- [AdaCL:Adaptive Continual Learning](https://mlsanity.com/paper/2303.13113/) (2023)
- [Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning](https://mlsanity.com/paper/2311.18266/) (2023)
- [Hyperparameter Selection in Continual Learning](https://mlsanity.com/paper/2404.06466/) (2024)
- [Active View Selection with Perturbed Gaussian Ensemble for Tomographic Reconstruction](https://mlsanity.com/paper/2603.06852/) (2026)
- [AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation](https://mlsanity.com/paper/2606.31204/) (2026)
- [What Can AutoML Do For Continual Learning?](https://mlsanity.com/paper/2311.11963/) (2023)
- [Ancestral Mamba: Enhancing Selective Discriminant Space Model with Online Visual Prototype Learning for Efficient and Robust Discriminant Approach](https://mlsanity.com/paper/2503.22729/) (2025)
- [RASteer: Retain-Aware Activation Steering for Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2610.01969/) (2026)
- [SATHUR: Self Augmenting Task Hallucinal Unified Representation for Generalized Class Incremental Learning](https://mlsanity.com/paper/2311.18630/) (2023)
- [Sample Weight Estimation Using Meta-Updates for Online Continual Learning](https://mlsanity.com/paper/2401.15973/) (2024)
- [ReFu: Recursive Fusion for Exemplar-Free 3D Class-Incremental Learning](https://mlsanity.com/paper/2409.12326/) (2024)
- [Index-Aligned Query Distillation for Transformer-based Incremental Object Detection](https://mlsanity.com/paper/2508.11339/) (2025)
- [Closing the Oracle Gap: Increment Vector Transformation for Class Incremental Learning](https://mlsanity.com/paper/2509.21898/) (2025)
- [DAO-GP Drift Aware Online Non-Linear Regression Gaussian-Process](https://mlsanity.com/paper/2512.08879/) (2025)
- [Policy Gradient with Adaptive Entropy Annealing for Continual Fine-Tuning](https://mlsanity.com/paper/2602.14078/) (2026)

## Abstract (arXiv)

Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. However, none of the existing CIL models can achieve the optimal tradeoff in different data-receiving settings--where typically the training-from-half (TFH) setting needs more stability, but the training-from-scratch (TFS) needs more plasticity. To this end, we design an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori. Specifically, we first introduce the key hyperparameters that influence the trade-off, e.g., knowledge distillation (KD) loss weights, learning rates, and classifier types. Then, we formulate the hyperparameter optimization process as an online Markov Decision Process (MDP) problem and propose a specific algorithm to solve it. We apply local estimated rewards and a classic bandit algorithm Exp3 to address the issues when applying online MDP methods to the CIL protocol. Our method consistently improves top-performing CIL methods in both TFH and TFS settings, e.g., boosting the average accuracy of TFH and TFS by 2.2 percentage points on ImageNet-Full, compared to the state-of-the-art.