# Fundamentals of Task-Agnostic Data Valuation

arXiv: https://arxiv.org/abs/2208.12354 · HTML: https://mlsanity.com/paper/2208.12354/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 25 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [A Principled Approach to Data Valuation for Federated Learning](https://mlsanity.com/paper/2009.06192/) (2020)
- [A Marketplace for Data: An Algorithmic Solution](https://mlsanity.com/paper/1805.08125/) (2018)
- [Collaborative Machine Learning with Incentive-Aware Model Rewards](https://mlsanity.com/paper/2010.12797/) (2020)
- [A Survey on Data Pricing: from Economics to Data Science](https://mlsanity.com/paper/2009.04462/) (2020)
- [Model-based Pricing for Machine Learning in a Data Marketplace](https://mlsanity.com/paper/1805.11450/) (2018)
- [Incentivizing Exploration with Selective Data Disclosure](https://mlsanity.com/paper/1811.06026/) (2018)
- [Incentivizing Collaboration in Machine Learning via Synthetic Data Rewards](https://mlsanity.com/paper/2112.09327/) (2021)
- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [Data Collection and Wireless Communication in Internet of Things (IoT) Using Economic Analysis and Pricing Models: A Survey](https://mlsanity.com/paper/1608.03475/) (2016)
- [Towards Data Auctions with Externalities](https://mlsanity.com/paper/2003.08345/) (2020)
- [Data Markets to support AI for All: Pricing, Valuation and Governance](https://mlsanity.com/paper/1905.06462/) (2019)
- [Data Sharing Markets](https://mlsanity.com/paper/2107.08630/) (2021)
- [Pareto-Improving Data-Sharing](https://mlsanity.com/paper/2205.11295/) (2022)
- [Optimal Advertising for Information Products](https://mlsanity.com/paper/2002.10045/) (2020)

## Built on by (most cited first, 19 of 19)

- [Data Shapley in One Training Run](https://mlsanity.com/paper/2406.11011/) (2024)
- [Capturing the Temporal Dependence of Training Data Influence](https://mlsanity.com/paper/2412.09538/) (2024)
- [Rethinking Data Shapley for Data Selection Tasks: Misleads and Merits](https://mlsanity.com/paper/2405.03875/) (2024)
- [DAVED: Data Acquisition via Experimental Design for Data Markets](https://mlsanity.com/paper/2403.13893/) (2024)
- [Data Distribution Valuation](https://mlsanity.com/paper/2410.04386/) (2024)
- [Data Similarity-Based One-Shot Clustering for Multi-Task Hierarchical Federated Learning](https://mlsanity.com/paper/2410.02733/) (2024)
- [RCC-PFL: Robust Client Clustering under Noisy Labels in Personalized Federated Learning](https://mlsanity.com/paper/2503.19886/) (2025)
- [Data Measurements for Decentralized Data Markets](https://mlsanity.com/paper/2406.04257/) (2024)
- [Data-Centric AI in the Age of Large Language Models](https://mlsanity.com/paper/2406.14473/) (2024)
- [Data Reliability Scoring](https://mlsanity.com/paper/2510.17085/) (2025)
- [FB-NLL: A Feature-Based Approach to Tackle Noisy Labels in Personalized Federated Learning](https://mlsanity.com/paper/2604.19729/) (2026)
- [Private, Augmentation-Robust and Task-Agnostic Data Valuation Approach for Data Marketplace](https://mlsanity.com/paper/2411.00745/) (2024)
- [Unifying and Optimizing Data Values for Selection via Sequential Decision-Making](https://mlsanity.com/paper/2502.04554/) (2025)
- [KAIROS: Scalable Model-Agnostic Data Valuation](https://mlsanity.com/paper/2506.23799/) (2025)
- [Pricing Strategies for Different Accuracy Models from the Same Dataset Based on Generalized Hotelling's Law](https://mlsanity.com/paper/2404.05272/) (2024)
- [Data Distribution Valuation Using Generalized Bayesian Inference](https://mlsanity.com/paper/2604.05993/) (2026)
- [QueryMarket: Cost-Aware Online Active Learning in Data Markets](https://mlsanity.com/paper/2606.17805/) (2026)
- [Eigenspace-Based Clustering for Personalized System Identification](https://mlsanity.com/paper/2606.20811/) (2026)
- [Efficient Decentralized Multi-task Dataset Valuation via Model Merging](https://mlsanity.com/paper/2607.03346/) (2026)

## Abstract (arXiv)

We study valuing the data of a data owner/seller for a data seeker/buyer. Data valuation is often carried out for a specific task assuming a particular utility metric, such as test accuracy on a validation set, that may not exist in practice. In this work, we focus on task-agnostic data valuation without any validation requirements. The data buyer has access to a limited amount of data (which could be publicly available) and seeks more data samples from a data seller. We formulate the problem as estimating the differences in the statistical properties of the data at the seller with respect to the baseline data available at the buyer. We capture these statistical differences through second moment by measuring diversity and relevance of the seller's data for the buyer; we estimate these measures through queries to the seller without requesting raw data. We design the queries with the proposed approach so that the seller is blind to the buyer's raw data and has no knowledge to fabricate responses to queries to obtain a desired outcome of the diversity and relevance trade-off.We will show through extensive experiments on real tabular and image datasets that the proposed estimates capture the diversity and relevance of the seller's data for the buyer.