# CBA: Improving Online Continual Learning via Continual Bias Adaptor

arXiv: https://arxiv.org/abs/2308.06925 · HTML: https://mlsanity.com/paper/2308.06925/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 14 Aug 2023.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Rainbow Memory: Continual Learning with a Memory of Diverse Samples](https://mlsanity.com/paper/2103.17230/) (2021)
- [Co2L: Contrastive Continual Learning](https://mlsanity.com/paper/2106.14413/) (2021)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning](https://mlsanity.com/paper/2103.13885/) (2021)
- [SS-IL: Separated Softmax for Incremental Learning](https://mlsanity.com/paper/2003.13947/) (2020)
- [Online Coreset Selection for Rehearsal-based Continual Learning](https://mlsanity.com/paper/2106.01085/) (2021)
- [Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation](https://mlsanity.com/paper/2204.00895/) (2022)
- [Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System](https://mlsanity.com/paper/2201.12604/) (2022)
- [Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference](https://mlsanity.com/paper/2110.10031/) (2021)
- [DenseNet: Implementing Efficient ConvNet Descriptor Pyramids](https://mlsanity.com/paper/1404.1869/) (2014)
- [AutoBalance: Optimized Loss Functions for Imbalanced Data](https://mlsanity.com/paper/2201.01212/) (2022)
- [A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal](https://mlsanity.com/paper/2209.13917/) (2022)

## Built on by (most cited first, 16 of 16)

- [SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning](https://mlsanity.com/paper/2501.13198/) (2025)
- [F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning](https://mlsanity.com/paper/2403.15751/) (2024)
- [DELTA: Decoupling Long-Tailed Online Continual Learning](https://mlsanity.com/paper/2404.04476/) (2024)
- [Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models](https://mlsanity.com/paper/2312.08888/) (2023)
- [Online Continual Learning via Logit Adjusted Softmax](https://mlsanity.com/paper/2311.06460/) (2023)
- [MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts](https://mlsanity.com/paper/2510.13500/) (2025)
- [Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task](https://mlsanity.com/paper/2405.17779/) (2024)
- [Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback](https://mlsanity.com/paper/2505.09925/) (2025)
- [Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective](https://mlsanity.com/paper/2408.13991/) (2024)
- [Theoretical Foundations of Continual Learning via Drift-Plus-Penalty](https://mlsanity.com/paper/2606.08452/) (2026)
- [ER-FSL: Experience Replay with Feature Subspace Learning for Online Continual Learning](https://mlsanity.com/paper/2407.12279/) (2024)
- [GPS: Distilling Compact Memories via Grid-based Patch Sampling for Efficient Online Class-Incremental Learning](https://mlsanity.com/paper/2504.10409/) (2025)
- [Multi-level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL](https://mlsanity.com/paper/2508.08677/) (2025)
- [Adaptive Model Ensemble for Continual Learning](https://mlsanity.com/paper/2509.19819/) (2025)
- [Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem](https://mlsanity.com/paper/2608.21539/) (2026)
- [Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time](https://mlsanity.com/paper/2609.36442/) (2026)

## Abstract (arXiv)

Online continual learning (CL) aims to learn new knowledge and consolidate previously learned knowledge from non-stationary data streams. Due to the time-varying training setting, the model learned from a changing distribution easily forgets the previously learned knowledge and biases toward the newly received task. To address this problem, we propose a Continual Bias Adaptor (CBA) module to augment the classifier network to adapt to catastrophic distribution change during training, such that the classifier network is able to learn a stable consolidation of previously learned tasks. In the testing stage, CBA can be removed which introduces no additional computation cost and memory overhead. We theoretically reveal the reason why the proposed method can effectively alleviate catastrophic distribution shifts, and empirically demonstrate its effectiveness through extensive experiments based on four rehearsal-based baselines and three public continual learning benchmarks.