# Semivalue-based data valuation is arbitrary and gameable

arXiv: https://arxiv.org/abs/2506.12619 · HTML: https://mlsanity.com/paper/2506.12619/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Builds on (most cited first, 24 of 24)

- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2110.14049/) (2021)
- [Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms](https://mlsanity.com/paper/1908.08619/) (2019)
- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [A Principled Approach to Data Valuation for Federated Learning](https://mlsanity.com/paper/2009.06192/) (2020)
- [A Distributional Framework for Data Valuation](https://mlsanity.com/paper/2002.12334/) (2020)
- [A Marketplace for Data: An Algorithmic Solution](https://mlsanity.com/paper/1805.08125/) (2018)
- [Bounding the Estimation Error of Sampling-based Shapley Value Approximation](https://mlsanity.com/paper/1306.4265/) (2013)
- [OpenDataVal: a Unified Benchmark for Data Valuation](https://mlsanity.com/paper/2306.10577/) (2023)
- [Algorithmic Fairness Datasets: the Story so Far](https://mlsanity.com/paper/2202.01711/) (2022)
- [Fairness Under Composition](https://mlsanity.com/paper/1806.06122/) (2018)
- [Data Valuation for Medical Imaging Using Shapley Value: Application on A Large-scale Chest X-ray Dataset](https://mlsanity.com/paper/2010.08006/) (2020)
- [Small Data, Big Decisions: Model Selection in the Small-Data Regime](https://mlsanity.com/paper/2009.12583/) (2020)
- [Fair Pipelines](https://mlsanity.com/paper/1707.00391/) (2017)
- [An Economic Solution to Copyright Challenges of Generative AI](https://mlsanity.com/paper/2404.13964/) (2024)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Variance reduced Shapley value estimation for trustworthy data valuation](https://mlsanity.com/paper/2210.16835/) (2022)
- [Private Data Valuation and Fair Payment in Data Marketplaces](https://mlsanity.com/paper/2210.08723/) (2022)
- [A Survey on Data Markets](https://mlsanity.com/paper/2411.07267/) (2024)
- [Towards More Efficient Data Valuation in Healthcare Federated Learning using Ensembling](https://mlsanity.com/paper/2209.05424/) (2022)
- [Abstracting Fairness: Oracles, Metrics, and Interpretability](https://mlsanity.com/paper/2004.01840/) (2020)
- [Data Shapley Value for Handling Noisy Labels: An application in Screening COVID-19 Pneumonia from Chest CT Scans](https://mlsanity.com/paper/2110.08726/) (2021)
- [Thresholding Data Shapley for Data Cleansing Using Multi-Armed Bandits](https://mlsanity.com/paper/2402.08209/) (2024)

## Built on by (most cited first, 1 of 1)

- [Shapley-based Data Valuation for LLM Alignment via Sequential Preference Optimization](https://mlsanity.com/paper/2512.15765/) (2025)

## Abstract (arXiv)

The game-theoretic notion of the semivalue offers a popular framework for credit attribution and data valuation in machine learning. Semivalues have been proposed for a variety of high-stakes decisions involving data, such as determining contributor compensation, acquiring data from external sources, or filtering out low-value datapoints. In these applications, semivalues depend on the specification of a utility function that maps subsets of data to a scalar score. While it is broadly agreed that this utility function arises from a composition of a learning algorithm and a performance metric, its actual instantiation involves numerous subtle modeling choices. We argue that this underspecification leads to varying degrees of arbitrariness in semivalue-based valuations. Small, but arguably reasonable changes to the utility function can induce substantial shifts in valuations across datapoints. Moreover, these valuation methodologies are also often gameable: low-cost adversarial strategies exist to exploit this ambiguity and systematically redistribute value among datapoints. Through theoretical constructions and empirical examples, we demonstrate that a bad-faith valuator can manipulate utility specifications to favor preferred datapoints, and that a good-faith valuator is left without principled guidance to justify any particular specification. These vulnerabilities raise ethical and epistemic concerns about the use of semivalues in several applications. We conclude by highlighting the burden of justification that semivalue-based approaches place on modelers and discuss important considerations for identifying appropriate uses.