Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry
Not on the map. Why?
Abstract (arXiv)
Deep learning's preponderance across scientific domains has reshaped high-stakes decision-making, making it essential to follow rigorous operational frameworks that include both Right-to-Privacy (RTP) and Right-to-Explanation (RTE). This paper examines the complexities of combining these two requirements. For RTP, we focus on `Differential privacy` (DP), which is considered the current gold standard for privacy-preserving machine learning due to its strong quantitative guarantee of privacy. For RTE, we focus on post-hoc explainers: they are the go-to option for model auditing as they operate independently of model training. We formally investigate DP models and various commonly-used post-hoc explainers: how to evaluate these explainers subject to RTP, and analyze the intrinsic interactions between DP models and these explainers. Furthermore, our work throws light on how RTP and RTE can be effectively combined in high-stakes applications. Our study concludes by outlining an industrial software pipeline, with the example of a wildly used use-case, that respects both RTP and RTE requirements.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 69 of its 110 references that are in the map. First posted on arXiv 30 Dec 2024.
Builds on
69 of 110 references.
References · 69
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2016Deep Learning with Differential Privacycited by 2,728
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
38 more
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
- 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
- 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
- 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
- 2019The many Shapley values for model explanationcited by 344
- 2016Top-down Neural Attention by Excitation Backpropcited by 321
- 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
- 2018On the Robustness of Interpretability Methodscited by 314
- 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
- 2018Scalable Private Learning with PATEcited by 269
- 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 239
- 2019Explainable Machine Learning in Deploymentcited by 211
- 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
- 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
- 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
- 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
- 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 153
- 2020A Survey of Privacy Attacks in Machine Learningcited by 146
- 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
- 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
- 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
- 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
- 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
- 2019Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Surveycited by 93
- 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
- 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
- 2020Aligning Faithful Interpretations with their Social Attributioncited by 79
- 2022Variational Model Inversion Attackscited by 75
- 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
- 2020On quantitative aspects of model interpretabilitycited by 71
- 2020Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorizationcited by 58
- 2018Algorithms that Remember: Model Inversion Attacks and Data Protection Lawcited by 55
- 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
- 2020Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualizationcited by 50
- 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
- 2020IROF: a low resource evaluation metric for explanation methodscited by 46
50 most cited of 69 All 69 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.