# Humans disagree with the IoU for measuring object detector localization error

arXiv: https://arxiv.org/abs/2207.14221 · HTML: https://mlsanity.com/paper/2207.14221/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 28 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [We don't need no bounding-boxes: Training object class detectors using only human verification](https://mlsanity.com/paper/1602.08405/) (2016)
- [Hallucination In Object Detection -- A Study In Visual Part Verification](https://mlsanity.com/paper/2106.02523/) (2021)
- [Robot In a Room: Toward Perfect Object Recognition in Closed Environments](https://mlsanity.com/paper/1507.02703/) (2015)
- [Evaluating Context for Deep Object Detectors](https://mlsanity.com/paper/2205.02887/) (2022)

## Built on by (most cited first, 4 of 4)

- [Rethinking Intersection Over Union for Small Object Detection in Few-Shot Regime](https://mlsanity.com/paper/2307.09562/) (2023)
- [Do Object Detection Localization Errors Affect Human Performance and Trust?](https://mlsanity.com/paper/2401.17821/) (2024)
- [Aligning Object Detector Bounding Boxes with Human Preference](https://mlsanity.com/paper/2408.10844/) (2024)
- [MicroscopyMatching: Towards a Ready-to-use Framework for Microscopy Image Analysis in Diverse Conditions](https://mlsanity.com/paper/2605.14980/) (2026)

## Abstract (arXiv)

The localization quality of automatic object detectors is typically evaluated by the Intersection over Union (IoU) score. In this work, we show that humans have a different view on localization quality. To evaluate this, we conduct a survey with more than 70 participants. Results show that for localization errors with the exact same IoU score, humans might not consider that these errors are equal, and express a preference. Our work is the first to evaluate IoU with humans and makes it clear that relying on IoU scores alone to evaluate localization errors might not be sufficient.