# Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review

arXiv: https://arxiv.org/abs/2411.05874 · HTML: https://mlsanity.com/paper/2411.05874/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 99 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies](https://mlsanity.com/paper/2007.15911/) (2020)
- [Post Hoc Explanations of Language Models Can Improve Language Models](https://mlsanity.com/paper/2305.11426/) (2023)
- [Interpret Federated Learning with Shapley Values](https://mlsanity.com/paper/1905.04519/) (2019)
- [Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability](https://mlsanity.com/paper/2111.10934/) (2021)
- [Designing ECG Monitoring Healthcare System with Federated Transfer Learning and Explainable AI](https://mlsanity.com/paper/2105.12497/) (2021)
- [Interpretable collaborative data analysis on distributed data](https://mlsanity.com/paper/2011.04437/) (2020)
- [Federated Learning in Practice: Reflections and Projections](https://mlsanity.com/paper/2410.08892/) (2024)
- [Towards Interpretable Federated Learning](https://mlsanity.com/paper/2302.13473/) (2023)
- [Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning](https://mlsanity.com/paper/2105.09540/) (2021)
- [Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data](https://mlsanity.com/paper/2203.01752/) (2022)
- [CXR-FL: Deep Learning-Based Chest X-ray Image Analysis Using Federated Learning](https://mlsanity.com/paper/2204.05203/) (2022)
- [Reward Systems for Trustworthy Medical Federated Learning](https://mlsanity.com/paper/2205.00470/) (2022)
- [Balancing Privacy Protection and Interpretability in Federated Learning](https://mlsanity.com/paper/2302.08044/) (2023)

## Built on by (most cited first, 6 of 6)

- [A Survey on Foundation Models for Personalized Federated Intelligence](https://mlsanity.com/paper/2505.06907/) (2025)
- [Decoding Federated Learning: The FedNAM+ Conformal Revolution](https://mlsanity.com/paper/2506.17872/) (2025)
- [Policy-Driven AI in Dataspaces: Taxonomy, Explainability, and Pathways for Compliant Innovation](https://mlsanity.com/paper/2507.20014/) (2025)
- [Hierarchical Federated Learning for Crop Yield Prediction in Smart Agricultural Production Systems](https://mlsanity.com/paper/2510.12727/) (2025)
- [DANCE: Dynamic, Available, Neighbor-gated Condensation for Federated Text-Attributed Graphs](https://mlsanity.com/paper/2601.16519/) (2026)
- [Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learning](https://mlsanity.com/paper/2609.03851/) (2026)

## Abstract (arXiv)

The joint implementation of federated learning (FL) and explainable artificial intelligence (XAI) could allow training models from distributed data and explaining their inner workings while preserving essential aspects of privacy. Toward establishing the benefits and tensions associated with their interplay, this scoping review maps the publications that jointly deal with FL and XAI, focusing on publications that reported an interplay between FL and model interpretability or post-hoc explanations. Out of the 37 studies meeting our criteria, only one explicitly and quantitatively analyzed the influence of FL on model explanations, revealing a significant research gap. The aggregation of interpretability metrics across FL nodes created generalized global insights at the expense of node-specific patterns being diluted. Several studies proposed FL algorithms incorporating explanation methods to safeguard the learning process against defaulting or malicious nodes. Studies using established FL libraries or following reporting guidelines are a minority. More quantitative research and structured, transparent practices are needed to fully understand their mutual impact and under which conditions it happens.