# Beyond performance-wise Contribution Evaluation in Federated Learning

arXiv: https://arxiv.org/abs/2602.22470 · HTML: https://mlsanity.com/paper/2602.22470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](https://mlsanity.com/paper/2012.10544/) (2020)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [Robustness of classifiers: from adversarial to random noise](https://mlsanity.com/paper/1608.08967/) (2016)
- [GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning](https://mlsanity.com/paper/2109.02053/) (2021)
- [A Systematic Study of Bias Amplification](https://mlsanity.com/paper/2201.11706/) (2022)
- [OpenDataVal: a Unified Benchmark for Data Valuation](https://mlsanity.com/paper/2306.10577/) (2023)
- [A Comprehensive Evaluation Framework for Deep Model Robustness](https://mlsanity.com/paper/2101.09617/) (2021)
- [Few-shot Backdoor Defense Using Shapley Estimation](https://mlsanity.com/paper/2112.14889/) (2021)
- [An Exploratory Analysis on Users' Contributions in Federated Learning](https://mlsanity.com/paper/2011.06830/) (2020)
- [Stabilizing Estimates of Shapley Values with Control Variates](https://mlsanity.com/paper/2310.07672/) (2023)
- [Towards Algorithmic Fairness by means of Instance-level Data Re-weighting based on Shapley Values](https://mlsanity.com/paper/2303.01928/) (2023)
- [FedSV: Byzantine-Robust Federated Learning via Shapley Value](https://mlsanity.com/paper/2502.17526/) (2025)
- [ACE: A Model Poisoning Attack on Contribution Evaluation Methods in Federated Learning](https://mlsanity.com/paper/2405.20975/) (2024)
- [Holistic Survey of Privacy and Fairness in Machine Learning](https://mlsanity.com/paper/2307.15838/) (2023)
- [Contribution Evaluation in Federated Learning: Examining Current Approaches](https://mlsanity.com/paper/2311.09856/) (2023)
- [Mechanism for Collaborative Federated Learning: Pitfalls of Shapley Values](https://mlsanity.com/paper/2403.04753/) (2024)
- [On the Fragility of Contribution Score Computation in Federated Learning](https://mlsanity.com/paper/2509.19921/) (2025)

## Abstract (arXiv)

Federated learning offers a privacy-friendly collaborative learning framework, yet its success, like any joint venture, hinges on the contributions of its participants. Existing client evaluation methods predominantly focus on model performance, such as accuracy or loss, which represents only one dimension of a machine learning model's overall utility. In contrast, this work investigates the critical, yet overlooked, issue of client contributions towards a model's trustworthiness -- specifically, its reliability (tolerance to noisy data), resilience (resistance to adversarial examples), and fairness (measured via demographic parity). To quantify these multifaceted contributions, we employ the state-of-the-art approximation of the Shapley value, a principled method for value attribution. Our results reveal that no single client excels across all dimensions, which are largely independent from each other, highlighting a critical flaw in current evaluation scheme: no single metric is adequate for comprehensive evaluation and equitable rewarding allocation.