mlsanityData to 7 Oct 2026

A systematic study of the foreground-background imbalance problem in deep learning for object detection

arXiv 2306.16539 ↗First posted 28 Jun 2023cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

The class imbalance problem in deep learning has been explored in several studies, but there has yet to be a systematic analysis of this phenomenon in object detection. Here, we present comprehensive analyses and experiments of the foreground-background (F-B) imbalance problem in object detection, which is very common and caused by small, infrequent objects of interest. We experimentally study the effects of different aspects of F-B imbalance (object size, number of objects, dataset size, object type) on detection performance. In addition, we also compare 9 leading methods for addressing this problem, including Faster-RCNN, SSD, OHEM, Libra-RCNN, Focal-Loss, GHM, PISA, YOLO-v3, and GFL with a range of datasets from different imaging domains. We conclude that (1) the F-B imbalance can indeed cause a significant drop in detection performance, (2) The detection performance is more affected by F-B imbalance when fewer training data are available, (3) in most cases, decreasing object size leads to larger performance drop than decreasing number of objects, given the same change in the ratio of object pixels to non-object pixels, (6) among all selected methods, Libra-RCNN and PISA demonstrate the best performance in addressing the issue of F-B imbalance. (7) When the training dataset size is large, the choice of method is not impactful (8) Soft-sampling methods, including focal-loss, GHM, and GFL, perform fairly well on average but are relatively unstable.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 47 references that are in the map. First posted on arXiv 28 Jun 2023.

Builds on

23 of 47 references.

References · 23

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  4. 2015Fast R-CNNcited by 2,930
  5. 2018YOLOv3: An Incremental Improvementcited by 2,652
  6. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  7. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  8. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  9. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  10. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 408
  11. 2020Detection and Tracking Meet Drones Challengecited by 261
  12. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
11 more
  1. 2019Imbalance Problems in Object Detection: A Reviewcited by 118
  2. 2018Gradient Harmonized Single-stage Detectorcited by 114
  3. 2017Perceptual Generative Adversarial Networks for Small Object Detectioncited by 76
  4. 2019Prime Sample Attention in Object Detectioncited by 55
  5. 2020Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Networkcited by 35
  6. 2019Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-based CT Image Augmentation for Object Detectioncited by 29
  7. 2019DR Loss: Improving Object Detection by Distributional Rankingcited by 24
  8. 2020AP-Loss for Accurate One-Stage Object Detectioncited by 24
  9. 2020Foreground-Background Imbalance Problem in Deep Object Detectors: A Reviewcited by 5
  10. 2019Residual Objectness for Imbalance Reductioncited by 3
  11. 2023Unsupervised anomaly localization in high-resolution breast scans using deep pluralistic image completioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Exponentially Weighted Instance-Aware Repeat Factor Sampling for Long-Tailed Object Detection Model Training in Unmanned Aerial Vehicles Surveillance Scenarioscited by 1
  2. 2025An Empirical Study on the Robustness of YOLO Models for Underwater Object Detectioncited by 0
  3. 2026Intelligent Detection of Mechanical, Electrical, and Plumbing (MEP) Metrics Based on 2D Floor Planscited by 0