# Gradient Reweighting: Towards Imbalanced Class-Incremental Learning

arXiv: https://arxiv.org/abs/2402.18528 · HTML: https://mlsanity.com/paper/2402.18528/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Adaptive Aggregation Networks for Class-Incremental Learning](https://mlsanity.com/paper/2010.05063/) (2020)
- [SS-IL: Separated Softmax for Incremental Learning](https://mlsanity.com/paper/2003.13947/) (2020)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classification](https://mlsanity.com/paper/2112.00412/) (2021)
- [Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection](https://mlsanity.com/paper/2012.08548/) (2020)
- [Influence-Balanced Loss for Imbalanced Visual Classification](https://mlsanity.com/paper/2110.02444/) (2021)

## Built on by (most cited first, 30 of 37)

- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning](https://mlsanity.com/paper/2505.24816/) (2025)
- [Multi-Granularity Class Prototype Topology Distillation for Class-Incremental Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2411.16064/) (2024)
- [AIR: Analytic Imbalance Rectifier for Continual Learning](https://mlsanity.com/paper/2408.10349/) (2024)
- [Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task](https://mlsanity.com/paper/2405.17779/) (2024)
- [Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2407.17120/) (2024)
- [IncreFA: Breaking the Static Wall of Generative Model Attribution](https://mlsanity.com/paper/2604.17736/) (2026)
- [Learning to Classify New Foods Incrementally Via Compressed Exemplars](https://mlsanity.com/paper/2404.07507/) (2024)
- [Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective](https://mlsanity.com/paper/2408.13991/) (2024)
- [Hierarchical Visual Prompt Learning for Continual Video Instance Segmentation](https://mlsanity.com/paper/2508.08612/) (2025)
- [Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging](https://mlsanity.com/paper/2512.01461/) (2025)
- [Boosting the Class-Incremental Learning in 3D Point Clouds via Zero-Collection-Cost Basic Shape Pre-Training](https://mlsanity.com/paper/2504.08412/) (2025)
- [Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning](https://mlsanity.com/paper/2510.03993/) (2025)
- [Can You Hear, Localize, and Segment Continually? An Exemplar-Free Continual Learning Benchmark for Audio-Visual Segmentation](https://mlsanity.com/paper/2603.08967/) (2026)
- [One Adapter for All: Towards Unified Representation in Step-Imbalanced Class-Incremental Learning](https://mlsanity.com/paper/2603.10237/) (2026)
- [Reframing Long-Tailed Learning via Loss Landscape Geometry](https://mlsanity.com/paper/2603.21217/) (2026)
- [Fair Class-Incremental Learning using Sample Weighting](https://mlsanity.com/paper/2410.01324/) (2024)
- [FocalCount: Towards Class-Count Imbalance in Class-Agnostic Counting](https://mlsanity.com/paper/2502.10677/) (2025)
- [Graph Neural Network-based Spectral Filtering Mechanism for Imbalance Classification in Network Digital Twin](https://mlsanity.com/paper/2502.11505/) (2025)
- [From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients](https://mlsanity.com/paper/2502.18762/) (2025)
- [Universal Incremental Learning: Mitigating Confusion from Inter- and Intra-task Distribution Randomness](https://mlsanity.com/paper/2503.07035/) (2025)
- [S2IL: Structurally Stable Incremental Learning](https://mlsanity.com/paper/2503.12193/) (2025)
- [ProtoGuard-guided PROPEL: Class-Aware Prototype Enhancement and Progressive Labeling for Incremental 3D Point Cloud Segmentation](https://mlsanity.com/paper/2504.01648/) (2025)
- [ViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning](https://mlsanity.com/paper/2507.17368/) (2025)
- [Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning](https://mlsanity.com/paper/2508.11353/) (2025)
- [Closing the Oracle Gap: Increment Vector Transformation for Class Incremental Learning](https://mlsanity.com/paper/2509.21898/) (2025)
- [EndoCIL: A Class-Incremental Learning Framework for Endoscopic Image Classification](https://mlsanity.com/paper/2510.17200/) (2025)
- [Pushing the Limits of Distillation-Based Continual Learning via Classifier-Proximal Lightweight Plugins](https://mlsanity.com/paper/2512.03537/) (2025)
- [Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning](https://mlsanity.com/paper/2603.08426/) (2026)
- [Dynamic Distillation and Gradient Consistency for Robust Long-Tailed Incremental Learning](https://mlsanity.com/paper/2605.03364/) (2026)

## Abstract (arXiv)

Class-Incremental Learning (CIL) trains a model to continually recognize new classes from non-stationary data while retaining learned knowledge. A major challenge of CIL arises when applying to real-world data characterized by non-uniform distribution, which introduces a dual imbalance problem involving (i) disparities between stored exemplars of old tasks and new class data (inter-phase imbalance), and (ii) severe class imbalances within each individual task (intra-phase imbalance). We show that this dual imbalance issue causes skewed gradient updates with biased weights in FC layers, thus inducing over/under-fitting and catastrophic forgetting in CIL. Our method addresses it by reweighting the gradients towards balanced optimization and unbiased classifier learning. Additionally, we observe imbalanced forgetting where paradoxically the instance-rich classes suffer higher performance degradation during CIL due to a larger amount of training data becoming unavailable in subsequent learning phases. To tackle this, we further introduce a distribution-aware knowledge distillation loss to mitigate forgetting by aligning output logits proportionally with the distribution of lost training data. We validate our method on CIFAR-100, ImageNetSubset, and Food101 across various evaluation protocols and demonstrate consistent improvements compared to existing works, showing great potential to apply CIL in real-world scenarios with enhanced robustness and effectiveness.