# Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection

arXiv: https://arxiv.org/abs/2104.05702 · HTML: https://mlsanity.com/paper/2104.05702/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Equalization Loss for Long-Tailed Object Recognition](https://mlsanity.com/paper/2003.05176/) (2020)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax](https://mlsanity.com/paper/2006.10408/) (2020)
- [Seesaw Loss for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2008.10032/) (2020)
- [Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.05830/) (2015)
- [Learning to Segment the Tail](https://mlsanity.com/paper/2004.00900/) (2020)
- [Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2008.05676/) (2020)

## Built on by (most cited first, 27 of 27)

- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Discovering Objects that Can Move](https://mlsanity.com/paper/2203.10159/) (2022)
- [On Model Calibration for Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.02170/) (2021)
- [BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training](https://mlsanity.com/paper/2203.13249/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization](https://mlsanity.com/paper/2206.11134/) (2022)
- [Relieving Long-tailed Instance Segmentation via Pairwise Class Balance](https://mlsanity.com/paper/2201.02784/) (2022)
- [Trapped in texture bias? A large scale comparison of deep instance segmentation](https://mlsanity.com/paper/2401.09109/) (2024)
- [Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition](https://mlsanity.com/paper/2410.06109/) (2024)
- [Open-Vocabulary Object Detection via Language Hierarchy](https://mlsanity.com/paper/2410.20371/) (2024)
- [Learning with Free Object Segments for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2202.11124/) (2022)
- [Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future](https://mlsanity.com/paper/2410.05577/) (2024)
- [Instance-Aware Repeat Factor Sampling for Long-Tailed Object Detection](https://mlsanity.com/paper/2305.08069/) (2023)
- [Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection](https://mlsanity.com/paper/2212.10147/) (2022)
- [TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributions](https://mlsanity.com/paper/2503.01924/) (2025)
- [Balanced Classification: A Unified Framework for Long-Tailed Object Detection](https://mlsanity.com/paper/2308.02213/) (2023)
- [Cannot See the Forest for the Trees: Aggregating Multiple Viewpoints to Better Classify Objects in Videos](https://mlsanity.com/paper/2206.02116/) (2022)
- [Semi-Supervised and Long-Tailed Object Detection with CascadeMatch](https://mlsanity.com/paper/2305.14813/) (2023)
- [Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Rate](https://mlsanity.com/paper/2304.00049/) (2023)
- [A Bilevel Optimization Framework for Imbalanced Data Classification](https://mlsanity.com/paper/2410.11171/) (2024)
- [TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2607.08201/) (2026)
- [Improving Long-tailed Object Detection with Image-Level Supervision by Multi-Task Collaborative Learning](https://mlsanity.com/paper/2210.05568/) (2022)
- [Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective](https://mlsanity.com/paper/2405.15157/) (2024)
- [Sampling Imbalanced Data with Multi-objective Bilevel Optimization](https://mlsanity.com/paper/2506.11315/) (2025)
- [Information Density Imbalance in Visual Object Detection](https://mlsanity.com/paper/2609.02369/) (2026)

## Abstract (arXiv)

Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resampling is typically introduced as a simple but effective approach. However, we observe that long-tailed detection differs from classification since multiple classes may be present in one image. As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level. We address object-level resampling by introducing an object-centric memory replay strategy based on dynamic, episodic memory banks. Our proposed strategy has two benefits: 1) convenient object-level resampling without significant extra computation, and 2) implicit feature-level augmentation from model updates. We show that image-level and object-level resamplings are both important, and thus unify them with a joint resampling strategy (RIO). Our method outperforms state-of-the-art long-tailed detection and segmentation methods on LVIS v0.5 across various backbones. Code is available at https://github.com/NVlabs/RIO.