# ICON2: Reliably Benchmarking Predictive Inequity in Object Detection

arXiv: https://arxiv.org/abs/2306.04482 · HTML: https://mlsanity.com/paper/2306.04482/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Perceptual Generative Adversarial Networks for Small Object Detection](https://mlsanity.com/paper/1706.05274/) (2017)
- [Scale Match for Tiny Person Detection](https://mlsanity.com/paper/1912.10664/) (2019)
- [Detection in Crowded Scenes: One Proposal, Multiple Predictions](https://mlsanity.com/paper/2003.09163/) (2020)
- [Deep Contrast Learning for Salient Object Detection](https://mlsanity.com/paper/1603.01976/) (2016)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models](https://mlsanity.com/paper/2110.15499/) (2021)
- [Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud](https://mlsanity.com/paper/2109.03285/) (2021)
- [Exploring Low-light Object Detection Techniques](https://mlsanity.com/paper/2107.14382/) (2021)
- [IMMVP: An Efficient Daytime and Nighttime On-Road Object Detector](https://mlsanity.com/paper/1910.06573/) (2019)
- [Mitigating Bias in Visual Transformers via Targeted Alignment](https://mlsanity.com/paper/2302.04358/) (2023)
- [The Problem of Fragmented Occlusion in Object Detection](https://mlsanity.com/paper/2004.13076/) (2020)

## Built on by (most cited first, 2 of 2)

- [FACET: Fairness in Computer Vision Evaluation Benchmark](https://mlsanity.com/paper/2309.00035/) (2023)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)

## Abstract (arXiv)

As computer vision systems are being increasingly deployed at scale in high-stakes applications like autonomous driving, concerns about social bias in these systems are rising. Analysis of fairness in real-world vision systems, such as object detection in driving scenes, has been limited to observing predictive inequity across attributes such as pedestrian skin tone, and lacks a consistent methodology to disentangle the role of confounding variables e.g. does my model perform worse for a certain skin tone, or are such scenes in my dataset more challenging due to occlusion and crowds? In this work, we introduce ICON2, a framework for robustly answering this question. ICON2 leverages prior knowledge on the deficiencies of object detection systems to identify performance discrepancies across sub-populations, compute correlations between these potential confounders and a given sensitive attribute, and control for the most likely confounders to obtain a more reliable estimate of model bias. Using our approach, we conduct an in-depth study on the performance of object detection with respect to income from the BDD100K driving dataset, revealing useful insights.