# Optimizing the Trade-off between Single-Stage and Two-Stage Object Detectors using Image Difficulty Prediction

arXiv: https://arxiv.org/abs/1803.08707 · HTML: https://mlsanity.com/paper/1803.08707/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2018.

## Built on by (most cited first, 18 of 18)

- [Comparing YOLOv8 and Mask R-CNN for instance segmentation in complex orchard environments](https://mlsanity.com/paper/2312.07935/) (2023)
- [Video-based fully automatic assessment of open surgery suturing skills](https://mlsanity.com/paper/2110.13972/) (2021)
- [A Lightweight Measure of Classification Difficulty from Application Dataset Characteristics](https://mlsanity.com/paper/2404.05981/) (2024)
- [Towards practical object detection for weed spraying in precision agriculture](https://mlsanity.com/paper/2109.11048/) (2021)
- [Open-Vocabulary Object Detection in UAV Imagery: A Review and Future Perspectives](https://mlsanity.com/paper/2507.13359/) (2025)
- [Deep adversarial attack on target detection systems](https://mlsanity.com/paper/2108.05948/) (2021)
- [LPYOLO: Low Precision YOLO for Face Detection on FPGA](https://mlsanity.com/paper/2207.10482/) (2022)
- [Sketch2FullStack: Generating Skeleton Code of Full Stack Website and Application from Sketch using Deep Learning and Computer Vision](https://mlsanity.com/paper/2211.14607/) (2022)
- [MLMT-CNN for Object Detection and Segmentation in Multi-layer and Multi-spectral Images](https://mlsanity.com/paper/2407.14473/) (2024)
- [SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video](https://mlsanity.com/paper/2203.15228/) (2022)
- [Critical Evaluation of LOCO dataset with Machine Learning](https://mlsanity.com/paper/2209.13499/) (2022)
- [Development, Optimization, and Deployment of Thermal Forward Vision Systems for Advance Vehicular Applications on Edge Devices](https://mlsanity.com/paper/2301.07613/) (2023)
- [Augmenting Zero-Shot Detection Training with Image Labels](https://mlsanity.com/paper/2306.06899/) (2023)
- [Voice-Assisted Real-Time Traffic Sign Recognition System Using Convolutional Neural Network](https://mlsanity.com/paper/2404.07807/) (2024)
- [AttentNet: Fully Convolutional 3D Attention for Lung Nodule Detection](https://mlsanity.com/paper/2407.14464/) (2024)
- [Improving Object Detection Performance through YOLOv8: A Comprehensive Training and Evaluation Study](https://mlsanity.com/paper/2505.11424/) (2025)
- [Efficient Neural Network Model Selection for Few-Class Application Datasets](https://mlsanity.com/paper/2606.19712/) (2026)
- [Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation](https://mlsanity.com/paper/2609.26549/) (2026)

## Abstract (arXiv)

There are mainly two types of state-of-the-art object detectors. On one hand, we have two-stage detectors, such as Faster R-CNN (Region-based Convolutional Neural Networks) or Mask R-CNN, that (i) use a Region Proposal Network to generate regions of interests in the first stage and (ii) send the region proposals down the pipeline for object classification and bounding-box regression. Such models reach the highest accuracy rates, but are typically slower. On the other hand, we have single-stage detectors, such as YOLO (You Only Look Once) and SSD (Singe Shot MultiBox Detector), that treat object detection as a simple regression problem by taking an input image and learning the class probabilities and bounding box coordinates. Such models reach lower accuracy rates, but are much faster than two-stage object detectors. In this paper, we propose to use an image difficulty predictor to achieve an optimal trade-off between accuracy and speed in object detection. The image difficulty predictor is applied on the test images to split them into easy versus hard images. Once separated, the easy images are sent to the faster single-stage detector, while the hard images are sent to the more accurate two-stage detector. Our experiments on PASCAL VOC 2007 show that using image difficulty compares favorably to a random split of the images. Our method is flexible, in that it allows to choose a desired threshold for splitting the images into easy versus hard.