# Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation

arXiv: https://arxiv.org/abs/2008.05676 · HTML: https://mlsanity.com/paper/2008.05676/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 30 of 41)

- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shot](https://mlsanity.com/paper/2108.02385/) (2021)
- [FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2102.12867/) (2021)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Exploring Classification Equilibrium in Long-Tailed Object Detection](https://mlsanity.com/paper/2108.07507/) (2021)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [On Model Calibration for Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.02170/) (2021)
- [Disentangle Your Dense Object Detector](https://mlsanity.com/paper/2107.02963/) (2021)
- [MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/2309.13042/) (2023)
- [MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection](https://mlsanity.com/paper/2102.08884/) (2021)
- [Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection](https://mlsanity.com/paper/2104.05702/) (2021)
- [Balancing Logit Variation for Long-tailed Semantic Segmentation](https://mlsanity.com/paper/2306.02061/) (2023)
- [Improving the Intra-class Long-tail in 3D Detection via Rare Example Mining](https://mlsanity.com/paper/2210.08375/) (2022)
- [Relieving Long-tailed Instance Segmentation via Pairwise Class Balance](https://mlsanity.com/paper/2201.02784/) (2022)
- [Long-tail Detection with Effective Class-Margins](https://mlsanity.com/paper/2301.09724/) (2023)
- [Long-tailed Instance Segmentation using Gumbel Optimized Loss](https://mlsanity.com/paper/2207.10936/) (2022)
- [A Comparative Review of Recent Few-Shot Object Detection Algorithms](https://mlsanity.com/paper/2111.00201/) (2021)
- [Open-Vocabulary Object Detection via Language Hierarchy](https://mlsanity.com/paper/2410.20371/) (2024)
- [Learning with Free Object Segments for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2202.11124/) (2022)
- [TS4Net: Two-Stage Sample Selective Strategy for Rotating Object Detection](https://mlsanity.com/paper/2108.03116/) (2021)
- [Inverse Image Frequency for Long-tailed Image Recognition](https://mlsanity.com/paper/2209.04861/) (2022)
- [KTN: Knowledge Transfer Network for Learning Multi-person 2D-3D Correspondences](https://mlsanity.com/paper/2206.10090/) (2022)
- [Balanced Classification: A Unified Framework for Long-Tailed Object Detection](https://mlsanity.com/paper/2308.02213/) (2023)
- [Rectify the Regression Bias in Long-Tailed Object Detection](https://mlsanity.com/paper/2401.15885/) (2024)
- [Fractal Calibration for long-tailed object detection](https://mlsanity.com/paper/2410.11774/) (2024)
- [BoundarySqueeze: Image Segmentation as Boundary Squeezing](https://mlsanity.com/paper/2105.11668/) (2021)
- [VLMine: Long-Tail Data Mining with Vision Language Models](https://mlsanity.com/paper/2409.15486/) (2024)

## Abstract (arXiv)

Despite the previous success of object analysis, detecting and segmenting a large number of object categories with a long-tailed data distribution remains a challenging problem and is less investigated. For a large-vocabulary classifier, the chance of obtaining noisy logits is much higher, which can easily lead to a wrong recognition. In this paper, we exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes, and construct a classification tree that is responsible for parsing an object instance into a fine-grained category via its parent class. In the classification tree, as the number of parent class nodes are significantly less, their logits are less noisy and can be utilized to suppress the wrong/noisy logits existed in the fine-grained class nodes. As the way to construct the parent class is not unique, we further build multiple trees to form a classification forest where each tree contributes its vote to the fine-grained classification. To alleviate the imbalanced learning caused by the long-tail phenomena, we propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models for recognizing more than 1000 categories. Extensive experiments are performed on the large vocabulary dataset LVIS. Compared with the Mask R-CNN baseline, the Forest R-CNN significantly boosts the performance with 11.5% and 3.9% AP improvements on the rare categories and overall categories, respectively. Moreover, we achieve state-of-the-art results on the LVIS dataset. Code is available at https://github.com/JialianW/Forest_RCNN.