Do Data Valuations Make Good Data Prices?
Not on the map. Why?
Abstract (arXiv)
As large language models increasingly rely on external data sources, compensating data contributors has become a central concern. But how should these payments be devised? We revisit data valuations from a textitmarket-design perspective where payments serve to compensate data owners for the textitprivate heterogeneous costs they incur for collecting and sharing data. We show that popular valuation methods-such as Leave-One-Out and Data Shapley-make for poor payments. They fail to ensure truthful reporting of the costs, leading to textitinefficient market outcomes. To address this, we adapt well-established payment rules from mechanism design, namely Myerson and Vickrey-Clarke-Groves (VCG), to the data market setting. We show that Myerson payment is the minimal truthful mechanism, optimal from the buyer's perspective. Additionally, we identify a condition under which both data buyers and sellers are utility-satisfied, and the market achieves efficiency. Our findings highlight the importance of incorporating incentive compatibility into data valuation design, paving the way for more robust and efficient data markets. Our data market framework is readily applicable to real-world scenarios. We illustrate this with simulations of contributor compensation in an LLM based retrieval-augmented generation (RAG) marketplace tasked with challenging medical question answering.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 40 references that are in the map. First posted on arXiv 7 Apr 2025.
Builds on
18 of 40 references.
References · 18
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
- 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
- 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
- 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 184
- 2019Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithmscited by 148
- 2019A Question-Entailment Approach to Question Answeringcited by 134
- 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 100
6 more
- 2024Data Shapley in One Training Runcited by 81
- 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 46
- 2020Efficient Two-Sided Markets with Limited Informationcited by 14
- 2020Towards Data Auctions with Externalitiescited by 9
- 2024A Survey on Data Marketscited by 7
- 2021Data Sharing Marketscited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Utility-Aware Data Pricing: Token-Level Quality and Empirical Training Gain for LLMs
- 2026The Economics of Model Collapse: Equilibrium, Welfare, and Optimal Provenance Subsidies in Synthetic Data Markets
- 2025Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value
- 2026The Price of Intelligence: A Quality-Adjusted Price Index for AI Services
- 2025Data Auctions for Retrieval Augmented Generation
- 2026The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More
- 2026Test-Time Compute Games
- 2026The Price of Ignorance: Information-Free Quotation for Data Retention in Machine Unlearning