# A Principled Approach to Data Valuation for Federated Learning

arXiv: https://arxiv.org/abs/2009.06192 · HTML: https://mlsanity.com/paper/2009.06192/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 30 of 127)

- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [Data Shapley in One Training Run](https://mlsanity.com/paper/2406.11011/) (2024)
- [GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning](https://mlsanity.com/paper/2109.02053/) (2021)
- [What Do We Mean by Generalization in Federated Learning?](https://mlsanity.com/paper/2110.14216/) (2021)
- [Towards Fairness-Aware Federated Learning](https://mlsanity.com/paper/2111.01872/) (2021)
- [Fair Federated Medical Image Segmentation via Client Contribution Estimation](https://mlsanity.com/paper/2303.16520/) (2023)
- [Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspective](https://mlsanity.com/paper/2111.11850/) (2021)
- [Mechanisms that Incentivize Data Sharing in Federated Learning](https://mlsanity.com/paper/2207.04557/) (2022)
- [Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value](https://mlsanity.com/paper/2304.07718/) (2023)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification](https://mlsanity.com/paper/2211.06800/) (2022)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Improving Fairness for Data Valuation in Horizontal Federated Learning](https://mlsanity.com/paper/2109.09046/) (2021)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [Proportional Fairness in Federated Learning](https://mlsanity.com/paper/2202.01666/) (2022)
- [Incentivizing Collaboration in Machine Learning via Synthetic Data Rewards](https://mlsanity.com/paper/2112.09327/) (2021)
- [Fundamentals of Task-Agnostic Data Valuation](https://mlsanity.com/paper/2208.12354/) (2022)
- [Industry-Scale Orchestrated Federated Learning for Drug Discovery](https://mlsanity.com/paper/2210.08871/) (2022)
- [Threshold KNN-Shapley: A Linear-Time and Privacy-Friendly Approach to Data Valuation](https://mlsanity.com/paper/2308.15709/) (2023)
- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [Fair and efficient contribution valuation for vertical federated learning](https://mlsanity.com/paper/2201.02658/) (2022)
- [Efficient Data Shapley for Weighted Nearest Neighbor Algorithms](https://mlsanity.com/paper/2401.11103/) (2024)
- [Vertical Federated Learning: A Structured Literature Review](https://mlsanity.com/paper/2212.00622/) (2022)
- [Data Valuation and Detections in Federated Learning](https://mlsanity.com/paper/2311.05304/) (2023)
- [Redefining Contributions: Shapley-Driven Federated Learning](https://mlsanity.com/paper/2406.00569/) (2024)
- [Transparent Contribution Evaluation for Secure Federated Learning on Blockchain](https://mlsanity.com/paper/2101.10572/) (2021)
- [Decentral and Incentivized Federated Learning Frameworks: A Systematic Literature Review](https://mlsanity.com/paper/2205.07855/) (2022)
- [FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning](https://mlsanity.com/paper/2106.02310/) (2021)

## Abstract (arXiv)

Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. In order to sustain long-term participation of data owners, it is important to fairly appraise each data source and compensate data owners for their contribution to the training process. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. It has been increasingly used for valuing training data in centralized learning. However, computing the SV requires exhaustively evaluating the model performance on every subset of data sources, which incurs prohibitive communication cost in the federated setting. Besides, the canonical SV ignores the order of data sources during training, which conflicts with the sequential nature of FL. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value. The federated SV preserves the desirable properties of the canonical SV while it can be calculated without incurring extra communication cost and is also able to capture the effect of participation order on data value. We conduct a thorough empirical study of the federated SV on a range of tasks, including noisy label detection, adversarial participant detection, and data summarization on different benchmark datasets, and demonstrate that it can reflect the real utility of data sources for FL and has the potential to enhance system robustness, security, and efficiency. We also report and analyze "failure cases" and hope to stimulate future research.