# On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

arXiv: https://arxiv.org/abs/2107.02170 · HTML: https://mlsanity.com/paper/2107.02170/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 76 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)

## Built on by (most cited first, 30 of 39)

- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion](https://mlsanity.com/paper/2212.03863/) (2022)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [Learning to Detect Mobile Objects from LiDAR Scans Without Labels](https://mlsanity.com/paper/2203.15882/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Fine-Tuning is Fine, if Calibrated](https://mlsanity.com/paper/2409.16223/) (2024)
- [Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization](https://mlsanity.com/paper/2206.11134/) (2022)
- [Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2403.11491/) (2024)
- [Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration](https://mlsanity.com/paper/2307.00934/) (2023)
- [Relieving Long-tailed Instance Segmentation via Pairwise Class Balance](https://mlsanity.com/paper/2201.02784/) (2022)
- [Calibrating Multimodal Learning](https://mlsanity.com/paper/2306.01265/) (2023)
- [Long-tailed Instance Segmentation using Gumbel Optimized Loss](https://mlsanity.com/paper/2207.10936/) (2022)
- [Estimating Model Performance under Domain Shifts with Class-Specific Confidence Scores](https://mlsanity.com/paper/2207.09957/) (2022)
- [Trapped in texture bias? A large scale comparison of deep instance segmentation](https://mlsanity.com/paper/2401.09109/) (2024)
- [Open-Vocabulary Object Detection via Language Hierarchy](https://mlsanity.com/paper/2410.20371/) (2024)
- [Learning with Free Object Segments for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2202.11124/) (2022)
- [Logit Normalization for Long-tail Object Detection](https://mlsanity.com/paper/2203.17020/) (2022)
- [The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition](https://mlsanity.com/paper/2210.05566/) (2022)
- [Taming the Long Tail of Deep Probabilistic Forecasting](https://mlsanity.com/paper/2202.13418/) (2022)
- [Beyond mAP: Towards better evaluation of instance segmentation](https://mlsanity.com/paper/2207.01614/) (2022)
- [Inverse Image Frequency for Long-tailed Image Recognition](https://mlsanity.com/paper/2209.04861/) (2022)
- [Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection](https://mlsanity.com/paper/2212.10147/) (2022)
- [Towards Heterogeneous Long-tailed Learning: Benchmarking, Metrics, and Toolbox](https://mlsanity.com/paper/2307.08235/) (2023)
- [Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks](https://mlsanity.com/paper/2407.08567/) (2024)
- [Fractal Calibration for long-tailed object detection](https://mlsanity.com/paper/2410.11774/) (2024)
- [BEV-LGKD: A Unified LiDAR-Guided Knowledge Distillation Framework for BEV 3D Object Detection](https://mlsanity.com/paper/2212.00623/) (2022)
- [Class Uncertainty: A Measure to Mitigate Class Imbalance](https://mlsanity.com/paper/2311.14090/) (2023)
- [Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2408.10537/) (2024)
- [V-MIND: Building Versatile Monocular Indoor 3D Detector with Diverse 2D Annotations](https://mlsanity.com/paper/2412.11412/) (2024)

## Abstract (arXiv)

Vanilla models for object detection and instance segmentation suffer from the heavy bias toward detecting frequent objects in the long-tailed setting. Existing methods address this issue mostly during training, e.g., by re-sampling or re-weighting. In this paper, we investigate a largely overlooked approach -- post-processing calibration of confidence scores. We propose NorCal, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size. We show that separately handling the background class and normalizing the scores over classes for each proposal are keys to achieving superior performance. On the LVIS dataset, NorCal can effectively improve nearly all the baseline models not only on rare classes but also on common and frequent classes. Finally, we conduct extensive analysis and ablation studies to offer insights into various modeling choices and mechanisms of our approach. Our code is publicly available at https://github.com/tydpan/NorCal/.