# IoU-balanced Loss Functions for Single-stage Object Detection

arXiv: https://arxiv.org/abs/1908.05641 · HTML: https://mlsanity.com/paper/1908.05641/

## Key facts

- Cited by 22 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 15 Aug 2019.

## Built on by (most cited first, 22 of 22)

- [One-Shot Object Affordance Detection in the Wild](https://mlsanity.com/paper/2108.03658/) (2021)
- [MS-DETR: Efficient DETR Training with Mixed Supervision](https://mlsanity.com/paper/2401.03989/) (2024)
- [Real-time Object Detection for Streaming Perception](https://mlsanity.com/paper/2203.12338/) (2022)
- [RGB Stream Is Enough for Temporal Action Detection](https://mlsanity.com/paper/2107.04362/) (2021)
- [Deep learning for table detection and structure recognition: A survey](https://mlsanity.com/paper/2211.08469/) (2022)
- [Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey](https://mlsanity.com/paper/2405.01725/) (2024)
- [StreamYOLO: Real-time Object Detection for Streaming Perception](https://mlsanity.com/paper/2207.10433/) (2022)
- [SCD: A Stacked Carton Dataset for Detection and Segmentation](https://mlsanity.com/paper/2102.12808/) (2021)
- [Revisiting AP Loss for Dense Object Detection: Adaptive Ranking Pair Selection](https://mlsanity.com/paper/2207.12042/) (2022)
- [Gaussian Guided IoU: A Better Metric for Balanced Learning on Object Detection](https://mlsanity.com/paper/2103.13613/) (2021)
- [Optimizing Data Processing in Space for Object Detection in Satellite Imagery](https://mlsanity.com/paper/2107.03774/) (2021)
- [Sign Language Recognition System using TensorFlow Object Detection API](https://mlsanity.com/paper/2201.01486/) (2022)
- [GMM: Delving into Gradient Aware and Model Perceive Depth Mining for Monocular 3D Detection](https://mlsanity.com/paper/2306.17450/) (2023)
- [Training Strategies for Isolated Sign Language Recognition](https://mlsanity.com/paper/2412.11553/) (2024)
- [Density-based Object Detection in Crowded Scenes](https://mlsanity.com/paper/2504.09819/) (2025)
- [Event-RGB Fusion for Spacecraft Pose Estimation Under Harsh Lighting](https://mlsanity.com/paper/2507.05698/) (2025)
- [Carton dataset synthesis method for domain shift based on foreground texture decoupling and replacement](https://mlsanity.com/paper/2103.10738/) (2021)
- [Layer-wise Customized Weak Segmentation Block and AIoU Loss for Accurate Object Detection](https://mlsanity.com/paper/2108.11021/) (2021)
- [Cascade RCNN for MIDOG Challenge](https://mlsanity.com/paper/2109.01085/) (2021)
- [Intelligent Debris Mass Estimation Model for Autonomous Underwater Vehicle](https://mlsanity.com/paper/2309.10617/) (2023)
- [Being Aware of Localization Accuracy By Generating Predicted-IoU-Guided Quality Scores](https://mlsanity.com/paper/2309.13269/) (2023)
- [Inter-Class Relational Loss for Small Object Detection: A Case Study on License Plates](https://mlsanity.com/paper/2508.14343/) (2025)

## Abstract (arXiv)

Single-stage object detectors have been widely applied in computer vision applications due to their high efficiency. However, we find that the loss functions adopted by single-stage object detectors hurt the localization accuracy seriously. Firstly, the standard cross-entropy loss for classification is independent of the localization task and drives all the positive examples to learn as high classification scores as possible regardless of localization accuracy during training. As a result, there will be many detections that have high classification scores but low IoU or detections that have low classification scores but high IoU. Secondly, for the standard smooth L1 loss, the gradient is dominated by the outliers that have poor localization accuracy during training. The above two problems will decrease the localization accuracy of single-stage detectors. In this work, IoU-balanced loss functions that consist of IoU-balanced classification loss and IoU-balanced localization loss are proposed to solve the above problems. The IoU-balanced classification loss pays more attention to positive examples with high IoU and can enhance the correlation between classification and localization tasks. The IoU-balanced localization loss decreases the gradient of examples with low IoU and increases the gradient of examples with high IoU, which can improve the localization accuracy of models. Extensive experiments on challenging public datasets such as MS COCO, PASCAL VOC and Cityscapes demonstrate that both IoU-balanced losses can bring substantial improvement for the popular single-stage detectors, especially for the localization accuracy. On COCO test-dev, the proposed methods can substantially improve AP by 1.0%~1.7% and AP75 by 1.0%~2.4%. On PASCAL VOC, it can also substantially improve AP by 1.3%~1.5% and AP80, AP90 by 1.6%~3.9%.