# Measuring the Impact of Scene Level Objects on Object Detection: Towards Quantitative Explanations of Detection Decisions

arXiv: https://arxiv.org/abs/2401.10790 · HTML: https://mlsanity.com/paper/2401.10790/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 11 of 11)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [Robust Object Detection under Occlusion with Context-Aware CompositionalNets](https://mlsanity.com/paper/2005.11643/) (2020)
- [Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation](https://mlsanity.com/paper/1812.06707/) (2018)
- [Explaining Autonomous Driving by Learning End-to-End Visual Attention](https://mlsanity.com/paper/2006.03347/) (2020)
- [Small Object Detection using Context and Attention](https://mlsanity.com/paper/1912.06319/) (2019)
- [Attentive Contexts for Object Detection](https://mlsanity.com/paper/1603.07415/) (2016)
- [White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks](https://mlsanity.com/paper/2104.02548/) (2021)
- [Exploring Person Context and Local Scene Context for Object Detection](https://mlsanity.com/paper/1511.08177/) (2015)

## Built on by (most cited first, 1 of 1)

- [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024)

## Abstract (arXiv)

Although accuracy and other common metrics can provide a useful window into the performance of an object detection model, they lack a deeper view of the model's decision process. Regardless of the quality of the training data and process, the features that an object detection model learns cannot be guaranteed. A model may learn a relationship between certain background context, i.e., scene level objects, and the presence of the labeled classes. Furthermore, standard performance verification and metrics would not identify this phenomenon. This paper presents a new black box explainability method for additional verification of object detection models by finding the impact of scene level objects on the identification of the objects within the image. By comparing the accuracies of a model on test data with and without certain scene level objects, the contributions of these objects to the model's performance becomes clearer. The experiment presented here will assess the impact of buildings and people in image context on the detection of emergency road vehicles by a fine-tuned YOLOv8 model. A large increase in accuracy in the presence of a scene level object will indicate the model's reliance on that object to make its detections. The results of this research lead to providing a quantitative explanation of the object detection model's decision process, enabling a deeper understanding of the model's performance.