# The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation

arXiv: https://arxiv.org/abs/2007.11978 · HTML: https://mlsanity.com/paper/2007.11978/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 30 of 108)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shot](https://mlsanity.com/paper/2108.02385/) (2021)
- [FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2102.12867/) (2021)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustment](https://mlsanity.com/paper/2305.11733/) (2023)
- [DELTA: degradation-free fully test-time adaptation](https://mlsanity.com/paper/2301.13018/) (2023)
- [Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.12940/) (2021)
- [Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering](https://mlsanity.com/paper/2207.12647/) (2022)
- [Adaptive Class Suppression Loss for Long-Tail Object Detection](https://mlsanity.com/paper/2104.00885/) (2021)
- [Exploring Classification Equilibrium in Long-Tailed Object Detection](https://mlsanity.com/paper/2108.07507/) (2021)
- [X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion](https://mlsanity.com/paper/2212.03863/) (2022)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2112.04564/) (2021)
- [On Model Calibration for Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.02170/) (2021)
- [BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation](https://mlsanity.com/paper/2204.02811/) (2022)
- [Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World](https://mlsanity.com/paper/2303.13233/) (2023)
- [MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/2309.13042/) (2023)
- [Invariant Feature Learning for Generalized Long-Tailed Classification](https://mlsanity.com/paper/2207.09504/) (2022)
- [MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection](https://mlsanity.com/paper/2102.08884/) (2021)
- [From Forks to Forceps: A New Framework for Instance Segmentation of Surgical Instruments](https://mlsanity.com/paper/2211.16200/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Class-Balanced Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2104.05279/) (2021)
- [ProCo: Prototype-aware Contrastive Learning for Long-tailed Medical Image Classification](https://mlsanity.com/paper/2209.00183/) (2022)
- [Semantic Segmentation In-the-Wild Without Seeing Any Segmentation Examples](https://mlsanity.com/paper/2112.03185/) (2021)
- [Class-Difficulty Based Methods for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2207.14499/) (2022)

## Abstract (arXiv)

Most existing object instance detection and segmentation models only work well on fairly balanced benchmarks where per-category training sample numbers are comparable, such as COCO. They tend to suffer performance drop on realistic datasets that are usually long-tailed. This work aims to study and address such open challenges. Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals. Based on such an observation, we first consider various techniques for improving long-tail classification performance which indeed enhance instance segmentation results. We then propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach. Without bells and whistles, it significantly boosts the performance of instance segmentation for tail classes on the recent LVIS dataset and our sampled COCO-LT dataset. Our analysis provides useful insights for solving long-tail instance detection and segmentation problems, and the straightforward SimCal method can serve as a simple but strong baseline. With the method we have won the 2019 LVIS challenge. Codes and models are available at https://github.com/twangnh/SimCal.