# Data Valuation for Medical Imaging Using Shapley Value: Application on A Large-scale Chest X-ray Dataset

arXiv: https://arxiv.org/abs/2010.08006 · HTML: https://mlsanity.com/paper/2010.08006/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 30 of 34)

- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology](https://mlsanity.com/paper/2206.01794/) (2022)
- [OpenDataVal: a Unified Benchmark for Data Valuation](https://mlsanity.com/paper/2306.10577/) (2023)
- [Fair Federated Medical Image Segmentation via Client Contribution Estimation](https://mlsanity.com/paper/2303.16520/) (2023)
- [Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value](https://mlsanity.com/paper/2304.07718/) (2023)
- [CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification](https://mlsanity.com/paper/2211.06800/) (2022)
- [Rethinking Data Shapley for Data Selection Tasks: Misleads and Merits](https://mlsanity.com/paper/2405.03875/) (2024)
- [VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels](https://mlsanity.com/paper/2102.11467/) (2021)
- [Variance reduced Shapley value estimation for trustworthy data valuation](https://mlsanity.com/paper/2210.16835/) (2022)
- [Shapley Values-enabled Progressive Pseudo Bag Augmentation for Whole Slide Image Classification](https://mlsanity.com/paper/2312.05490/) (2023)
- [Evaluating the Values of Sources in Transfer Learning](https://mlsanity.com/paper/2104.12567/) (2021)
- [A Marketplace for Trading AI Models based on Blockchain and Incentives for IoT Data](https://mlsanity.com/paper/2112.02870/) (2021)
- [2D-Shapley: A Framework for Fragmented Data Valuation](https://mlsanity.com/paper/2306.10473/) (2023)
- [Towards More Efficient Data Valuation in Healthcare Federated Learning using Ensembling](https://mlsanity.com/paper/2209.05424/) (2022)
- [Value of Information and Timing-aware Scheduling for Federated Learning](https://mlsanity.com/paper/2312.10512/) (2023)
- [Rethinking Robustness of Model Attributions](https://mlsanity.com/paper/2312.10534/) (2023)
- [Data Valuation with Gradient Similarity](https://mlsanity.com/paper/2405.08217/) (2024)
- [Understanding Data Influence with Differential Approximation](https://mlsanity.com/paper/2508.14648/) (2025)
- [EcoVal: An Efficient Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2402.09288/) (2024)
- [Uncertainty Quantification of Data Shapley via Statistical Inference](https://mlsanity.com/paper/2407.19373/) (2024)
- [Data Shapley Value for Handling Noisy Labels: An application in Screening COVID-19 Pneumonia from Chest CT Scans](https://mlsanity.com/paper/2110.08726/) (2021)
- [Data valuation: The partial ordinal Shapley value for machine learning](https://mlsanity.com/paper/2305.01660/) (2023)
- [Towards Data Valuation via Asymmetric Data Shapley](https://mlsanity.com/paper/2411.00388/) (2024)
- [On the Impact of the Utility in Semivalue-based Data Valuation](https://mlsanity.com/paper/2502.06574/) (2025)
- [Semivalue-based data valuation is arbitrary and gameable](https://mlsanity.com/paper/2506.12619/) (2025)
- [Scalable Data Point Valuation in Decentralized Learning](https://mlsanity.com/paper/2305.01657/) (2023)
- [Optimizing Product Provenance Verification using Data Valuation Methods](https://mlsanity.com/paper/2502.15177/) (2025)
- [A case for data valuation transparency via DValCards](https://mlsanity.com/paper/2506.23349/) (2025)
- [Hierarchical Dataset Selection for High-Quality Data Sharing](https://mlsanity.com/paper/2512.10952/) (2025)

## Abstract (arXiv)

The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in a large chest X-ray dataset. We characterized the effectiveness of data Shapley in identifying low quality versus valuable data for pneumonia detection. We found that removing training data with high Shapley values decreased the pneumonia detection performance, whereas removing data with low Shapley values improved the model performance. Furthermore, there were more mislabeled examples in low Shapley value data and more true pneumonia cases in high Shapley value data. Our results suggest that low Shapley value indicates mislabeled or poor quality images, whereas high Shapley value indicates data that are valuable for pneumonia detection. Our method can serve as a framework for using data Shapley to denoise large-scale medical imaging datasets.