# Hallucination In Object Detection -- A Study In Visual Part Verification

arXiv: https://arxiv.org/abs/2106.02523 · HTML: https://mlsanity.com/paper/2106.02523/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Object detection via a multi-region & semantic segmentation-aware CNN model](https://mlsanity.com/paper/1505.01749/) (2015)
- [Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships](https://mlsanity.com/paper/1807.00119/) (2018)
- [CoupleNet: Coupling Global Structure with Local Parts for Object Detection](https://mlsanity.com/paper/1708.02863/) (2017)
- [Shift Equivariance in Object Detection](https://mlsanity.com/paper/2008.05787/) (2020)
- [Exploring the Bounds of the Utility of Context for Object Detection](https://mlsanity.com/paper/1711.05471/) (2017)
- [Domain Adaptation with Joint Learning for Generic, Optical Car Part Recognition and Detection Systems (Go-CaRD)](https://mlsanity.com/paper/2006.08521/) (2020)

## Built on by (most cited first, 16 of 16)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models](https://mlsanity.com/paper/2310.05338/) (2023)
- [GROUNDHOG: Grounding Large Language Models to Holistic Segmentation](https://mlsanity.com/paper/2402.16846/) (2024)
- [Treble Counterfactual VLMs: A Causal Approach to Hallucination](https://mlsanity.com/paper/2503.06169/) (2025)
- [A Survey of Multimodal Hallucination Evaluation and Detection](https://mlsanity.com/paper/2507.19024/) (2025)
- [VIPriors 2: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2201.08625/) (2022)
- [Evaluating Context for Deep Object Detectors](https://mlsanity.com/paper/2205.02887/) (2022)
- [Humans disagree with the IoU for measuring object detector localization error](https://mlsanity.com/paper/2207.14221/) (2022)
- [VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2305.19688/) (2023)
- ['Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube](https://mlsanity.com/paper/2203.04837/) (2022)
- [Bent & Broken Bicycles: Leveraging synthetic data for damaged object re-identification](https://mlsanity.com/paper/2304.07883/) (2023)
- [An Effective Two-stage Training Paradigm Detector for Small Dataset](https://mlsanity.com/paper/2309.05652/) (2023)
- [VIPriors 4: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2406.18176/) (2024)
- [Aligning Object Detector Bounding Boxes with Human Preference](https://mlsanity.com/paper/2408.10844/) (2024)
- [Data-Efficient Challenges in Visual Inductive Priors: A Retrospective](https://mlsanity.com/paper/2506.08612/) (2025)
- [The 3D Mirage: Probing and Taming 3D Hallucinations](https://mlsanity.com/paper/2512.15423/) (2025)

## Abstract (arXiv)

We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.