# Interpret Federated Learning with Shapley Values

arXiv: https://arxiv.org/abs/1905.04519 · HTML: https://mlsanity.com/paper/1905.04519/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2019.

## Built on by (most cited first, 30 of 33)

- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [VeriFi: Towards Verifiable Federated Unlearning](https://mlsanity.com/paper/2205.12709/) (2022)
- [Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value](https://mlsanity.com/paper/2304.07718/) (2023)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directions](https://mlsanity.com/paper/2208.03392/) (2022)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges](https://mlsanity.com/paper/2102.01375/) (2021)
- [FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler](https://mlsanity.com/paper/2309.14675/) (2023)
- [Towards Interpretable Federated Learning](https://mlsanity.com/paper/2302.13473/) (2023)
- [Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond](https://mlsanity.com/paper/2207.09087/) (2022)
- [FederatedTrust: A Solution for Trustworthy Federated Learning](https://mlsanity.com/paper/2302.09844/) (2023)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures](https://mlsanity.com/paper/2404.00673/) (2024)
- [Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review](https://mlsanity.com/paper/2411.05874/) (2024)
- [IFedAvg: Interpretable Data-Interoperability for Federated Learning](https://mlsanity.com/paper/2107.06580/) (2021)
- [Hijack Vertical Federated Learning Models As One Party](https://mlsanity.com/paper/2212.00322/) (2022)
- [Proof-of-Contribution-Based Design for Collaborative Machine Learning on Blockchain](https://mlsanity.com/paper/2302.14031/) (2023)
- [Privacy-Preserving Algorithmic Recourse](https://mlsanity.com/paper/2311.14137/) (2023)
- [A Survey on Contribution Evaluation in Vertical Federated Learning](https://mlsanity.com/paper/2405.02364/) (2024)
- [Exploring Data Redundancy in Real-world Image Classification through Data Selection](https://mlsanity.com/paper/2306.14113/) (2023)
- [Federated and Transfer Learning for Cancer Detection Based on Image Analysis](https://mlsanity.com/paper/2405.20126/) (2024)
- [Tree-based Models for Vertical Federated Learning: A Survey](https://mlsanity.com/paper/2504.02285/) (2025)
- [Local Model Reconstruction Attacks in Federated Learning and their Uses](https://mlsanity.com/paper/2210.16205/) (2022)
- [Towards Bridging the FL Performance-Explainability Trade-Off: A Trustworthy 6G RAN Slicing Use-Case](https://mlsanity.com/paper/2307.12903/) (2023)
- [A Comprehensive Study of Shapley Value in Data Analytics](https://mlsanity.com/paper/2412.01460/) (2024)
- [Label Inference Attacks against Node-level Vertical Federated GNNs](https://mlsanity.com/paper/2308.02465/) (2023)
- [Game of Privacy: Towards Better Federated Platform Collaboration under Privacy Restriction](https://mlsanity.com/paper/2202.05139/) (2022)
- [Achieving Transparency in Distributed Machine Learning with Explainable Data Collaboration](https://mlsanity.com/paper/2212.03373/) (2022)
- [The Landscape of Modern Machine Learning: A Review of Machine, Distributed and Federated Learning](https://mlsanity.com/paper/2312.03120/) (2023)

## Abstract (arXiv)

Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the data are not disclosed in between each party. In this paper we investigate the model interpretation methods for Federated Learning, specifically on the measurement of feature importance of vertical Federated Learning where feature space of the data is divided into two parties, namely host and guest. For host party to interpret a single prediction of vertical Federated Learning model, the interpretation results, namely the feature importance, are very likely to reveal the protected data from guest party. We propose a method to balance the model interpretability and data privacy in vertical Federated Learning by using Shapley values to reveal detailed feature importance for host features and a unified importance value for federated guest features. Our experiments indicate robust and informative results for interpreting Federated Learning models.