# Relieving Long-tailed Instance Segmentation via Pairwise Class Balance

arXiv: https://arxiv.org/abs/2201.02784 · HTML: https://mlsanity.com/paper/2201.02784/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 52 references that are in the map.
- First posted on arXiv 8 Jan 2022.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Long-tailed Recognition by Routing Diverse Distribution-Aware Experts](https://mlsanity.com/paper/2010.01809/) (2020)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [Rethinking the Value of Labels for Improving Class-Imbalanced Learning](https://mlsanity.com/paper/2006.07529/) (2020)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)

## Built on by (most cited first, 18 of 18)

- [ReCo: Retrieve and Co-segment for Zero-shot Transfer](https://mlsanity.com/paper/2206.07045/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Logit Normalization for Long-tail Object Detection](https://mlsanity.com/paper/2203.17020/) (2022)
- [RAHNet: Retrieval Augmented Hybrid Network for Long-tailed Graph Classification](https://mlsanity.com/paper/2308.02335/) (2023)
- [Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks](https://mlsanity.com/paper/2407.08567/) (2024)
- [Balanced Classification: A Unified Framework for Long-Tailed Object Detection](https://mlsanity.com/paper/2308.02213/) (2023)
- [Rectify the Regression Bias in Long-Tailed Object Detection](https://mlsanity.com/paper/2401.15885/) (2024)
- [Fractal Calibration for long-tailed object detection](https://mlsanity.com/paper/2410.11774/) (2024)
- [Zero-shot Unsupervised Transfer Instance Segmentation](https://mlsanity.com/paper/2304.14376/) (2023)
- [Semi-Supervised and Long-Tailed Object Detection with CascadeMatch](https://mlsanity.com/paper/2305.14813/) (2023)
- [Better (pseudo-)labels for semi-supervised instance segmentation](https://mlsanity.com/paper/2403.11675/) (2024)
- [Long-Tailed Object Detection Pre-training: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction](https://mlsanity.com/paper/2411.09453/) (2024)
- [CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation](https://mlsanity.com/paper/2502.17429/) (2025)
- [Generalized Category Discovery under the Long-Tailed Distribution](https://mlsanity.com/paper/2506.12515/) (2025)
- [TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2607.08201/) (2026)
- [Improving Long-tailed Object Detection with Image-Level Supervision by Multi-Task Collaborative Learning](https://mlsanity.com/paper/2210.05568/) (2022)
- [PAT: Pixel-wise Adaptive Training for Long-tailed Segmentation](https://mlsanity.com/paper/2404.05393/) (2024)
- [Cost-Sensitive Learning for Long-Tailed Temporal Action Segmentation](https://mlsanity.com/paper/2503.18358/) (2025)

## Abstract (arXiv)

Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. This renders "how to appropriately define and alleviate the bias" one of the most important issues. Prior works mainly use label distribution or mean score information to indicate a coarse-grained bias. In this paper, we explore to excavate the confusion matrix, which carries the fine-grained misclassification details, to relieve the pairwise biases, generalizing the coarse one. To this end, we propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences. PCB generates fightback soft labels for regularization during training. Besides, an iterative learning paradigm is developed to support a progressive and smooth regularization in such debiasing. PCB can be plugged and played to any existing method as a complement. Experimental results on LVIS demonstrate that our method achieves state-of-the-art performance without bells and whistles. Superior results across various architectures show the generalization ability. The code and trained models are available at https://github.com/megvii-research/PCB.