# Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models

arXiv: https://arxiv.org/abs/2304.03218 · HTML: https://mlsanity.com/paper/2304.03218/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 6 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [CheXclusion: Fairness gaps in deep chest X-ray classifiers](https://mlsanity.com/paper/2003.00827/) (2020)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information](https://mlsanity.com/paper/1709.01447/) (2017)
- [Debiasing Skin Lesion Datasets and Models? Not So Fast](https://mlsanity.com/paper/2004.11457/) (2020)
- [(De)Constructing Bias on Skin Lesion Datasets](https://mlsanity.com/paper/1904.08818/) (2019)
- [Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts](https://mlsanity.com/paper/2009.10132/) (2020)
- [Measuring Model Biases in the Absence of Ground Truth](https://mlsanity.com/paper/2103.03417/) (2021)
- [Skin Deep Unlearning: Artefact and Instrument Debiasing in the Context of Melanoma Classification](https://mlsanity.com/paper/2109.09818/) (2021)
- [Evaluating Fairness Metrics in the Presence of Dataset Bias](https://mlsanity.com/paper/1809.09245/) (2018)
- [Algorithmic encoding of protected characteristics in image-based models for disease detection](https://mlsanity.com/paper/2110.14755/) (2021)

## Built on by (most cited first, 4 of 4)

- [Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling](https://mlsanity.com/paper/2502.09688/) (2025)
- [Causality-Driven Audits of Model Robustness](https://mlsanity.com/paper/2410.23494/) (2024)
- [A Causal Framework for Aligning Image Quality Metrics and Deep Neural Network Robustness](https://mlsanity.com/paper/2503.02797/) (2025)
- [Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Framework](https://mlsanity.com/paper/2503.09969/) (2025)

## Abstract (arXiv)

To safely deploy deep learning-based computer vision models for computer-aided detection and diagnosis, we must ensure that they are robust and reliable. Towards that goal, algorithmic auditing has received substantial attention. To guide their audit procedures, existing methods rely on heuristic approaches or high-level objectives (e.g., non-discrimination in regards to protected attributes, such as sex, gender, or race). However, algorithms may show bias with respect to various attributes beyond the more obvious ones, and integrity issues related to these more subtle attributes can have serious consequences. To enable the generation of actionable, data-driven hypotheses which identify specific dataset attributes likely to induce model bias, we contribute a first technique for the rigorous, quantitative screening of medical image datasets. Drawing from literature in the causal inference and information theory domains, our procedure decomposes the risks associated with dataset attributes in terms of their detectability and utility (defined as the amount of information knowing the attribute gives about a task label). To demonstrate the effectiveness and sensitivity of our method, we develop a variety of datasets with synthetically inserted artifacts with different degrees of association to the target label that allow evaluation of inherited model biases via comparison of performance against true counterfactual examples. Using these datasets and results from hundreds of trained models, we show our screening method reliably identifies nearly imperceptible bias-inducing artifacts. Lastly, we apply our method to the natural attributes of a popular skin-lesion dataset and demonstrate its success. Our approach provides a means to perform more systematic algorithmic audits and guide future data collection efforts in pursuit of safer and more reliable models.