SoK: Privacy Attacks on Machine Learning via Explainable AI
Not on the map. Why?
Abstract (arXiv)
Machine learning explanations reveal model behavior beyond predictions, creating attack surfaces for model confidentiality and data privacy. We systematize 25 studies that exploit explanations for model extraction, membership inference, and model inversion, treating attribute inference as partial inversion. Existing work is often labeled only black- or white-box, obscuring substantial differences in what explanation signal reaches an adversary. We therefore separate model knowledge from explanation acquisition and identify five paths: target-released, attacker-derived, secondary disclosure, privileged access, and released global artifacts. Across these paths, explanations reduce extraction cost, expose membership signals through explanation statistics, recourse distance, and explanation-guided robustness, and support spatial or algebraic reconstruction of private inputs. We compare system and threat models, explanation signals, auxiliary knowledge, target models, modalities, query budgets, evaluation metrics, reported performance, and defenses. Our analysis shows that no explanation family is uniformly unsafe and no defense is uniformly effective. Risk depends on which signal is exposed, how it is acquired, which asset is targeted, and what the attacker already knows. We argue that explanation privacy should therefore be evaluated as an end-to-end disclosure problem, with defenses matched to the acquisition path and protected asset.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 57 references that are in the map. First posted on arXiv 9 Sep 2026.
Builds on
33 of 57 references.
References · 33
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2016Deep Learning with Differential Privacycited by 2,728
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
- 2019Deep Leakage from Gradientscited by 1,021
- 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
- 2020iDLG: Improved Deep Leakage from Gradientscited by 382
- 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
- 2018PRADA: Protecting against DNN Model Stealing Attackscited by 165
- 2020A Survey of Privacy Attacks in Machine Learningcited by 146
21 more
- 2018Model Reconstruction from Model Explanationscited by 86
- 2021Exploiting Explanations for Model Inversion Attackscited by 49
- 2020Model extraction from counterfactual explanationscited by 47
- 2021MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AIcited by 35
- 2022DualCF: Efficient Model Extraction Attack from Counterfactual Explanationscited by 32
- 2022On the Privacy Risks of Algorithmic Recoursecited by 32
- 2020Model Explanations with Differential Privacycited by 23
- 2022Membership Inference Attack Using Self Influence Functionscited by 20
- 2022SoK: Explainable Machine Learning for Computer Security Applicationscited by 20
- 2022The privacy issue of counterfactual explanations: explanation linkage attackscited by 20
- 2022Inferring Sensitive Attributes from Model Explanationscited by 15
- 2022XRand: Differentially Private Defense against Explanation-Guided Attackscited by 11
- 2023AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Modelscited by 9
- 2024Feature Inference Attack on Shapley Valuescited by 9
- 2025From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attackscited by 3
- 2025Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Reviewcited by 2
- 2026Linear Model Extraction via Factual and Counterfactual Queriescited by 2
- 2024Knowledge Distillation-Based Model Extraction Attack using GAN-based Private Counterfactual Explanationscited by 1
- 2025Models That Are Interpretable But Not Transparentcited by 1
- 2026DeepLeak: Privacy Enhancing Hardening of Model Explanations Against Membership Leakagecited by 1
- 2026Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defensecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Towards a Game-theoretic Understanding of Explanation-based Membership Inference Attacks
- 2024Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference
- 2025Beyond Verification: Abductive Explanations for Post-AI Assessment of Privacy Leakage
- 2026TIER: Trajectory-Invariant Explanation Regularization for Membership Privacy
- 2022Lessons Learned: Defending Against Property Inference Attacks
- 2023Students Parrot Their Teachers: Membership Inference on Model Distillation
- 2024Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks
- 2025Evaluating the Defense Potential of Machine Unlearning against Membership Inference Attacks