Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Review
Not on the map. Why?
Abstract (arXiv)
Explainable Artificial Intelligence (XAI) has emerged as a pillar of Trustworthy AI and aims to bring transparency in complex models that are opaque by nature. Despite the benefits of incorporating explanations in models, an urgent need is found in addressing the privacy concerns of providing this additional information to end users. In this article, we conduct a scoping review of existing literature to elicit details on the conflict between privacy and explainability. Using the standard methodology for scoping review, we extracted 57 articles from 1,943 studies published from January 2019 to December 2024. The review addresses 3 research questions to present readers with more understanding of the topic: (1) what are the privacy risks of releasing explanations in AI systems? (2) what current methods have researchers employed to achieve privacy preservation in XAI systems? (3) what constitutes a privacy preserving explanation? Based on the knowledge synthesized from the selected studies, we categorize the privacy risks and preservation methods in XAI and propose the characteristics of privacy preserving explanations to aid researchers and practitioners in understanding the requirements of XAI that is privacy compliant. Lastly, we identify the challenges in balancing privacy with other system desiderata and provide recommendations for achieving privacy preserving XAI. We expect that this review will shed light on the complex relationship of privacy and explainability, both being the fundamental principles of Trustworthy AI.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 68 of its 184 references that are in the map. First posted on arXiv 5 May 2025.
Builds on
68 of 184 references.
References · 68
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
- 2014Striving for Simplicity: The All Convolutional Netcited by 990
38 more
- 2021Membership Inference Attacks From First Principlescited by 878
- 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
- 2020Label-Only Membership Inference Attackscited by 388
- 2019The many Shapley values for model explanationcited by 344
- 2023Explainability for Large Language Models: A Surveycited by 333
- 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
- 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 297
- 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 262
- 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
- 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
- 2017Machine Learning Models that Remember Too Muchcited by 169
- 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
- 2020A Survey of Privacy Attacks in Machine Learningcited by 146
- 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
- 2020Drug discovery with explainable artificial intelligencecited by 118
- 2019XRAI: Better Attributions Through Regionscited by 116
- 2023Low-Cost High-Power Membership Inference Attackscited by 107
- 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
- 2020Privacy Preserving Vertical Federated Learning for Tree-based Modelscited by 89
- 2018Model Reconstruction from Model Explanationscited by 86
- 2019Measure Contribution of Participants in Federated Learningcited by 79
- 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 76
- 2018Algorithms that Remember: Model Inversion Attacks and Data Protection Lawcited by 55
- 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
- 2020The Pragmatic Turn in Explainable Artificial Intelligence (XAI)cited by 54
- 2021Exploiting Explanations for Model Inversion Attackscited by 49
- 2020Model extraction from counterfactual explanationscited by 47
- 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
- 2021Survey: Leakage and Privacy at Inference Timecited by 44
- 2021MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AIcited by 35
- 2022DualCF: Efficient Model Extraction Attack from Counterfactual Explanationscited by 32
- 2022On the Privacy Risks of Algorithmic Recoursecited by 32
- 2024On the Privacy Risk of In-context Learningcited by 32
- 2019Interpretable and Differentially Private Predictionscited by 28
- 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
- 2021Accuracy, Interpretability, and Differential Privacy via Explainable Boostingcited by 24
- 2022Membership Inference Attack Using Self Influence Functionscited by 20
- 2022The privacy issue of counterfactual explanations: explanation linkage attackscited by 20
50 most cited of 68 All 68 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.