On the Volatility of Shapley-Based Contribution Metrics in Federated Learning
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) is a collaborative and privacy-preserving Machine Learning paradigm, allowing the development of robust models without the need to centralize sensitive data. A critical challenge in FL lies in fairly and accurately allocating contributions from diverse participants. Inaccurate allocation can undermine trust, lead to unfair compensation, and thus participants may lack the incentive to join or actively contribute to the federation. Various remuneration strategies have been proposed to date, including auction-based approaches and Shapley-value-based methods, the latter offering a means to quantify the contribution of each participant. However, little to no work has studied the stability of these contribution evaluation methods. In this paper, we evaluate participant contributions in federated learning using gradient-based model reconstruction techniques with Shapley values and compare the round-based contributions to a classic data contribution measurement scheme. We provide an extensive analysis of the discrepancies of Shapley values across a set of aggregation strategies and examine them on an overall and a per-client level. We show that, between different aggregation techniques, Shapley values lead to unstable reward allocations among participants. Our analysis spans various data heterogeneity distributions, including independent and identically distributed (IID) and non-IID scenarios.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 13 May 2024.
Builds on
25 of 38 references.
References · 25
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2020Adaptive Federated Optimizationcited by 1,036
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
- 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
- 2019Fair Resource Allocation in Federated Learningcited by 389
- 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
13 more
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2020A Principled Approach to Data Valuation for Federated Learningcited by 127
- 2021FastSHAP: Real-Time Shapley Value Estimationcited by 123
- 2020Learning to Detect Malicious Clients for Robust Federated Learningcited by 86
- 2019Measure Contribution of Participants in Federated Learningcited by 78
- 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 78
- 2021GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learningcited by 73
- 2021Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspectivecited by 47
- 2019Motivating Workers in Federated Learning: A Stackelberg Game Perspectivecited by 45
- 2021A Comprehensive Survey of Incentive Mechanism for Federated Learningcited by 38
- 2020Hierarchically Fair Federated Learningcited by 28
- 2019Rewarding High-Quality Data via Influence Functionscited by 11
Built on by
Citing papers per year: 2026: 2.
2 papers.