# Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

arXiv: https://arxiv.org/abs/2001.01385 · HTML: https://mlsanity.com/paper/2001.01385/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2020.

## Built on by (most cited first, 30 of 76)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Imbalance Trouble: Revisiting Neural-Collapse Geometry](https://mlsanity.com/paper/2208.05512/) (2022)
- [Label-Imbalanced and Group-Sensitive Classification under Overparameterization](https://mlsanity.com/paper/2103.01550/) (2021)
- [Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2203.15359/) (2022)
- [Adversarial Robustness under Long-Tailed Distribution](https://mlsanity.com/paper/2104.02703/) (2021)
- [AutoBalance: Optimized Loss Functions for Imbalanced Data](https://mlsanity.com/paper/2201.01212/) (2022)
- [On Model Calibration for Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.02170/) (2021)
- [Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspective](https://mlsanity.com/paper/2111.03874/) (2021)
- [MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection](https://mlsanity.com/paper/2102.08884/) (2021)
- [Understanding Why Generalized Reweighting Does Not Improve Over ERM](https://mlsanity.com/paper/2201.12293/) (2022)
- [Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data](https://mlsanity.com/paper/2212.13827/) (2022)
- [Subclass-balancing Contrastive Learning for Long-tailed Recognition](https://mlsanity.com/paper/2306.15925/) (2023)
- [Procrustean Training for Imbalanced Deep Learning](https://mlsanity.com/paper/2104.01769/) (2021)
- [A Unified Perspective for Loss-Oriented Imbalanced Learning via Localization](https://mlsanity.com/paper/2310.04752/) (2023)
- [Fine-Tuning is Fine, if Calibrated](https://mlsanity.com/paper/2409.16223/) (2024)
- [On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data](https://mlsanity.com/paper/2303.07608/) (2023)
- [Open Long-Tailed Recognition in a Dynamic World](https://mlsanity.com/paper/2208.08349/) (2022)
- [Balanced Product of Calibrated Experts for Long-Tailed Recognition](https://mlsanity.com/paper/2206.05260/) (2022)
- [Is Importance Weighting Incompatible with Interpolating Classifiers?](https://mlsanity.com/paper/2112.12986/) (2021)
- [Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Model](https://mlsanity.com/paper/2401.02058/) (2024)
- [Delving into the Trajectory Long-tail Distribution for Muti-object Tracking](https://mlsanity.com/paper/2403.04700/) (2024)
- [Towards Calibrated Hyper-Sphere Representation via Distribution Overlap Coefficient for Long-tailed Learning](https://mlsanity.com/paper/2208.10043/) (2022)
- [On Bridging Generic and Personalized Federated Learning for Image Classification](https://mlsanity.com/paper/2107.00778/) (2021)
- [DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Variations](https://mlsanity.com/paper/2405.07780/) (2024)
- [Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data](https://mlsanity.com/paper/2502.10381/) (2025)
- [NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.16709/) (2023)
- [DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets](https://mlsanity.com/paper/2404.02900/) (2024)
- [Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts](https://mlsanity.com/paper/2507.07100/) (2025)
- [Wrapped Cauchy Distributed Angular Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2305.18732/) (2023)
- [A Survey of Deep Long-Tail Classification Advancements](https://mlsanity.com/paper/2404.15593/) (2024)

## Abstract (arXiv)

Classifiers trained with class-imbalanced data are known to perform poorly on test data of the "minor" classes, of which we have insufficient training data. In this paper, we investigate learning a ConvNet classifier under such a scenario. We found that a ConvNet significantly over-fits the minor classes, which is quite opposite to traditional machine learning algorithms that often under-fit minor classes. We conducted a series of analysis and discovered the feature deviation phenomenon -- the learned ConvNet generates deviated features between the training and test data of minor classes -- which explains how over-fitting happens. To compensate for the effect of feature deviation which pushes test data toward low decision value regions, we propose to incorporate class-dependent temperatures (CDT) in training a ConvNet. CDT simulates feature deviation in the training phase, forcing the ConvNet to enlarge the decision values for minor-class data so that it can overcome real feature deviation in the test phase. We validate our approach on benchmark datasets and achieve promising performance. We hope that our insights can inspire new ways of thinking in resolving class-imbalanced deep learning.