# Towards Measuring Bias in Image Classification

arXiv: https://arxiv.org/abs/2107.00360 · HTML: https://mlsanity.com/paper/2107.00360/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 10 of 10)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Synthetic Data for Deep Learning](https://mlsanity.com/paper/1909.11512/) (2019)
- [Benchmarking Attribution Methods with Relative Feature Importance](https://mlsanity.com/paper/1907.09701/) (2019)
- [Examining CNN Representations with respect to Dataset Bias](https://mlsanity.com/paper/1710.10577/) (2017)

## Built on by (most cited first, 6 of 6)

- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [IFBiD: Inference-Free Bias Detection](https://mlsanity.com/paper/2109.04374/) (2021)
- [A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis](https://mlsanity.com/paper/2210.04491/) (2022)
- [Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision](https://mlsanity.com/paper/2607.07236/) (2026)
- [A survey on bias in machine learning research](https://mlsanity.com/paper/2308.11254/) (2023)
- [Towards Exploring Fairness in Visual Transformer based Natural and GAN Image Detection Systems](https://mlsanity.com/paper/2310.12076/) (2023)

## Abstract (arXiv)

Convolutional Neural Networks (CNN) have become de fact state-of-the-art for the main computer vision tasks. However, due to the complex underlying structure their decisions are hard to understand which limits their use in some context of the industrial world. A common and hard to detect challenge in machine learning (ML) tasks is data bias. In this work, we present a systematic approach to uncover data bias by means of attribution maps. For this purpose, first an artificial dataset with a known bias is created and used to train intentionally biased CNNs. The networks' decisions are then inspected using attribution maps. Finally, meaningful metrics are used to measure the attribution maps' representativeness with respect to the known bias. The proposed study shows that some attribution map techniques highlight the presence of bias in the data better than others and metrics can support the identification of bias.