# Uncertainty Quantification of Data Shapley via Statistical Inference

arXiv: https://arxiv.org/abs/2407.19373 · HTML: https://mlsanity.com/paper/2407.19373/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 28 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2110.14049/) (2021)
- [Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms](https://mlsanity.com/paper/1908.08619/) (2019)
- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [Machine Unlearning: Solutions and Challenges](https://mlsanity.com/paper/2308.07061/) (2023)
- [A Distributional Framework for Data Valuation](https://mlsanity.com/paper/2002.12334/) (2020)
- [A Survey on Data Pricing: from Economics to Data Science](https://mlsanity.com/paper/2009.04462/) (2020)
- [Model-based Pricing for Machine Learning in a Data Marketplace](https://mlsanity.com/paper/1805.11450/) (2018)
- [Data Valuation for Medical Imaging Using Shapley Value: Application on A Large-scale Chest X-ray Dataset](https://mlsanity.com/paper/2010.08006/) (2020)
- [An Economic Solution to Copyright Challenges of Generative AI](https://mlsanity.com/paper/2404.13964/) (2024)
- [Variance reduced Shapley value estimation for trustworthy data valuation](https://mlsanity.com/paper/2210.16835/) (2022)
- [Revenue Maximization for Query Pricing](https://mlsanity.com/paper/1909.00845/) (2019)
- [Approximating high-dimensional infinite-order U-statistics: statistical and computational guarantees](https://mlsanity.com/paper/1901.01163/) (2019)
- [Online Pricing with Reserve Price Constraint for Personal Data Markets](https://mlsanity.com/paper/1911.12598/) (2019)

## Built on by (most cited first, 2 of 2)

- [Faithful Group Shapley Value](https://mlsanity.com/paper/2505.19013/) (2025)
- [Towards Data Valuation via Asymmetric Data Shapley](https://mlsanity.com/paper/2411.00388/) (2024)

## Abstract (arXiv)

As data plays an increasingly pivotal role in decision-making, the emergence of data markets underscores the growing importance of data valuation. Within the machine learning landscape, Data Shapley stands out as a widely embraced method for data valuation. However, a limitation of Data Shapley is its assumption of a fixed dataset, contrasting with the dynamic nature of real-world applications where data constantly evolves and expands. This paper establishes the relationship between Data Shapley and infinite-order U-statistics and addresses this limitation by quantifying the uncertainty of Data Shapley with changes in data distribution from the perspective of U-statistics. We make statistical inferences on data valuation to obtain confidence intervals for the estimations. We construct two different algorithms to estimate this uncertainty and provide recommendations for their applicable situations. We also conduct a series of experiments on various datasets to verify asymptotic normality and propose a practical trading scenario enabled by this method.