# Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning

arXiv: https://arxiv.org/abs/2508.11353 · HTML: https://mlsanity.com/paper/2508.11353/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 70 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Gradient Harmonized Single-stage Detector](https://mlsanity.com/paper/1811.05181/) (2018)
- [Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection](https://mlsanity.com/paper/2012.08548/) (2020)
- [DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data](https://mlsanity.com/paper/2105.02340/) (2021)
- [Gradient Reweighting: Towards Imbalanced Class-Incremental Learning](https://mlsanity.com/paper/2402.18528/) (2024)
- [A Systematic Study of Online Class Imbalance Learning with Concept Drift](https://mlsanity.com/paper/1703.06683/) (2017)
- [Adaptive Cost-sensitive Online Classification](https://mlsanity.com/paper/1804.02246/) (2018)
- [Machine Learning-Based Heart Disease Diagnosis: A Systematic Literature Review](https://mlsanity.com/paper/2112.06459/) (2021)
- [Online Ensemble Learning for Imbalanced Data Streams](https://mlsanity.com/paper/1310.8004/) (2013)
- [Gradient Harmonization in Unsupervised Domain Adaptation](https://mlsanity.com/paper/2408.00288/) (2024)

## Abstract (arXiv)

Many real-world data are sequentially collected over time and often exhibit skewed class distributions, resulting in imbalanced data streams. While existing approaches have explored several strategies, such as resampling and reweighting, for imbalanced data stream learning, our work distinguishes itself by addressing the imbalance problem through training modification, particularly focusing on gradient descent techniques. We introduce the harmonized gradient descent (HGD) algorithm, which aims to equalize the norms of gradients across different classes. By ensuring the gradient norm balance, HGD mitigates under-fitting for minor classes and achieves balanced online learning. Notably, HGD operates in a streamlined implementation process, requiring no data-buffer, extra parameters, or prior knowledge, making it applicable to any learning models utilizing gradient descent for optimization. Theoretical analysis, based on a few common and mild assumptions, shows that HGD achieves a satisfied sub-linear regret bound. The proposed algorithm are compared with the commonly used online imbalance learning methods under several imbalanced data stream scenarios. Extensive experimental evaluations demonstrate the efficiency and effectiveness of HGD in learning imbalanced data streams.