# Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax

arXiv: https://arxiv.org/abs/2006.10408 · HTML: https://mlsanity.com/paper/2006.10408/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 30 of 152)

- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model](https://mlsanity.com/paper/2203.14940/) (2022)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space](https://mlsanity.com/paper/2105.01879/) (2021)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning](https://mlsanity.com/paper/2110.05474/) (2021)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition](https://mlsanity.com/paper/2107.09249/) (2021)
- [Learning Affordance Grounding from Exocentric Images](https://mlsanity.com/paper/2203.09905/) (2022)
- [Visual Alignment Constraint for Continuous Sign Language Recognition](https://mlsanity.com/paper/2104.02330/) (2021)
- [FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2102.12867/) (2021)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Self Supervision to Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2109.04075/) (2021)
- [Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2203.15359/) (2022)
- [Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details](https://mlsanity.com/paper/2102.01066/) (2021)
- [Adaptive Class Suppression Loss for Long-Tail Object Detection](https://mlsanity.com/paper/2104.00885/) (2021)
- [Exploring Classification Equilibrium in Long-Tailed Object Detection](https://mlsanity.com/paper/2108.07507/) (2021)
- [On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors](https://mlsanity.com/paper/2103.12474/) (2021)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [Tracking Every Thing in the Wild](https://mlsanity.com/paper/2207.12978/) (2022)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2112.04564/) (2021)
- [Injecting Semantic Concepts into End-to-End Image Captioning](https://mlsanity.com/paper/2112.05230/) (2021)
- [Towards Long-Tailed 3D Detection](https://mlsanity.com/paper/2211.08691/) (2022)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)

## Abstract (arXiv)

Solving long-tail large vocabulary object detection with deep learning based models is a challenging and demanding task, which is however under-explored.In this work, we provide the first systematic analysis on the underperformance of state-of-the-art models in front of long-tail distribution. We find existing detection methods are unable to model few-shot classes when the dataset is extremely skewed, which can result in classifier imbalance in terms of parameter magnitude. Directly adapting long-tail classification models to detection frameworks can not solve this problem due to the intrinsic difference between detection and classification.In this work, we propose a novel balanced group softmax (BAGS) module for balancing the classifiers within the detection frameworks through group-wise training. It implicitly modulates the training process for the head and tail classes and ensures they are both sufficiently trained, without requiring any extra sampling for the instances from the tail classes.Extensive experiments on the very recent long-tail large vocabulary object recognition benchmark LVIS show that our proposed BAGS significantly improves the performance of detectors with various backbones and frameworks on both object detection and instance segmentation. It beats all state-of-the-art methods transferred from long-tail image classification and establishes new state-of-the-art.Code is available at https://github.com/FishYuLi/BalancedGroupSoftmax.