# ContextShift: A Controlled Benchmark for Context Dependence in Object Detection

arXiv: https://arxiv.org/abs/2606.09495 · HTML: https://mlsanity.com/paper/2606.09495/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Modeling Visual Context is Key to Augmenting Object Detection Datasets](https://mlsanity.com/paper/1807.07428/) (2018)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [A Survey on Bias in Visual Datasets](https://mlsanity.com/paper/2107.07919/) (2021)
- [Black-box Explanation of Object Detectors via Saliency Maps](https://mlsanity.com/paper/2006.03204/) (2020)
- [D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement](https://mlsanity.com/paper/2410.13842/) (2024)
- [Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases](https://mlsanity.com/paper/2303.05470/) (2023)
- [Role of Spatial Context in Adversarial Robustness for Object Detection](https://mlsanity.com/paper/1910.00068/) (2019)
- [COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts](https://mlsanity.com/paper/2307.12730/) (2023)
- [Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation](https://mlsanity.com/paper/2402.18919/) (2024)
- [CLAD: A Contrastive Learning based Approach for Background Debiasing](https://mlsanity.com/paper/2210.02748/) (2022)
- [Impact of Data Duplication on Deep Neural Network-Based Image Classifiers: Robust vs. Standard Models](https://mlsanity.com/paper/2504.00638/) (2025)
- [Quantifying Context Bias in Domain Adaptation for Object Detection](https://mlsanity.com/paper/2409.14679/) (2024)

## Abstract (arXiv)

Modern object detectors achieve strong performance on standard benchmarks, yet their robustness to contextual variation remains insufficiently understood. Prior evaluations largely rely on aggregate metrics such as AP on uncontrolled distribution shifts, which can obscure how performance degrades under context change. We introduce ContextShift, a controlled benchmark that systematically manipulates object--context relationships while preserving object appearance. Built on COCO 2017, it isolates context as an independent variable through geometric transformations and synthetic and natural background substitutions, including a continuous compatibility axis based on normalized pointwise mutual information (NPMI). Across diverse detector architectures, we observe a consistent degradation pattern: false negatives increase by up to 227% and prediction volume decreases by up to 44%, while false positives remain stable or decline. This suppression behavior is not captured by aggregate metrics such as AP, which can mask substantial recall loss and changes in prediction dynamics. Further analysis suggests that degradation is driven less by reduced confidence than by a reduced formation of valid detection candidates. Moreover, performance along the statistical compatibility axis is non-monotonic, peaking at intermediate NPMI and degrading toward both extremes, indicating that statistical co-occurrence does not correlate linearly with effective visual context. Finally, we show that context-aware augmentation improves robustness: every augmented variant outperforms the dataset-only baseline on both original and manipulated test images, partially recovering performance lost to prediction-suppression failures by exposing models to object--context decoupling during training.