mlsanityData to 14 Oct 2026

Data Shapley: Equitable Valuation of Data for Machine Learning

arXiv 1904.02868 ↗First posted 5 Apr 2019stat.ML cs.AI cs.LGcited by 629

Not on the map. Why?

Cited by 21 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As data becomes the fuel driving technological and economic growth, a fundamental challenge is how to quantify the value of data in algorithmic predictions and decisions. For example, in healthcare and consumer markets, it has been suggested that individuals should be compensated for the data that they generate, but it is not clear what is an equitable valuation for individual data. In this work, we develop a principled framework to address data valuation in the context of supervised machine learning. Given a learning algorithm trained on n data points to produce a predictor, we propose data Shapley as a metric to quantify the value of each training datum to the predictor performance. Data Shapley value uniquely satisfies several natural properties of equitable data valuation. We develop Monte Carlo and gradient-based methods to efficiently estimate data Shapley values in practical settings where complex learning algorithms, including neural networks, are trained on large datasets. In addition to being equitable, extensive experiments across biomedical, image and synthetic data demonstrate that data Shapley has several other benefits: 1) it is more powerful than the popular leave-one-out or leverage score in providing insight on what data is more valuable for a given learning task; 2) low Shapley value data effectively capture outliers and corruptions; 3) high Shapley value data inform what type of new data to acquire to improve the predictor.

Key facts. Cited by 629 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 50; 2022: 66; 2023: 101; 2024: 126; 2025: 136; 2026: 150.

629 papers.

Sort:

By area:

  1. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  2. 2023Explainability for Large Language Models: A Surveycited by 333
  3. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  4. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  5. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  6. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  7. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 165
  8. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  9. 2022Data Banzhaf: A Robust Data Valuation Framework for Machine Learningcited by 132
  10. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 123
  11. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 121
  12. 2023Foundation Models and Fair Usecited by 116
38 more
  1. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  2. 2022The Shapley Value in Machine Learningcited by 98
  3. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 94
  4. 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 90
  5. 2023Data Distillation: A Surveycited by 87
  6. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  7. 2022MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflictscited by 81
  8. 2024Data Shapley in One Training Runcited by 81
  9. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 78
  10. 2021GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learningcited by 74
  11. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
  12. 2022Faith-Shap: The Faithful Shapley Interaction Indexcited by 69
  13. 2023LAVA: Data Valuation without Pre-Specified Learning Algorithmscited by 69
  14. 2023In-context Example Selection with Influencescited by 61
  15. 2022Adaptive Second Order Coresets for Data-efficient Machine Learningcited by 60
  16. 2022Data Curation Alone Can Stabilize In-context Learningcited by 59
  17. 2021Towards Fairness-Aware Federated Learningcited by 58
  18. 2022PUMA: Performance Unchanged Model Augmentation for Training Data Removalcited by 58
  19. 2023Diagnosing and Rectifying Vision Models using Languagecited by 58
  20. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  21. 2023OpenDataVal: a Unified Benchmark for Data Valuationcited by 55
  22. 2023Fair Federated Medical Image Segmentation via Client Contribution Estimationcited by 52
  23. 2023SHAP-IQ: Unified Approximation of any-order Shapley Interactionscited by 51
  24. 2022Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experimentscited by 50
  25. 2021Interactive Label Cleaning with Example-based Explanationscited by 49
  26. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  27. 2023Intriguing Properties of Data Attribution on Diffusion Modelscited by 47
  28. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 45
  29. 2023Evaluating Data Attribution for Text-to-Image Modelscited by 45
  30. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 43
  31. 2024shapiq: Shapley Interactions for Machine Learningcited by 43
  32. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 41
  33. 2021One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learningcited by 39
  34. 2021HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networkscited by 38
  35. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 38
  36. 2021A Comprehensive Survey of Incentive Mechanism for Federated Learningcited by 38
  37. 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
  38. 2023Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Valuecited by 38

50 most cited of 629 All 629 (JSON)