# ExpProof : Operationalizing Explanations for Confidential Models with ZKPs

arXiv: https://arxiv.org/abs/2502.03773 · HTML: https://mlsanity.com/paper/2502.03773/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 68 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [A survey of algorithmic recourse: definitions, formulations, solutions, and prospects](https://mlsanity.com/paper/2010.04050/) (2020)
- [Counterfactual Explanations Can Be Manipulated](https://mlsanity.com/paper/2106.02666/) (2021)
- [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020)
- [Fairwashing: the risk of rationalization](https://mlsanity.com/paper/1901.09749/) (2019)
- [Framework for Evaluating Faithfulness of Local Explanations](https://mlsanity.com/paper/2202.00734/) (2022)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [zkLLM: Zero Knowledge Proofs for Large Language Models](https://mlsanity.com/paper/2404.16109/) (2024)
- [Inverse Classification for Comparison-based Interpretability in Machine Learning](https://mlsanity.com/paper/1712.08443/) (2017)
- [Defining Locality for Surrogates in Post-hoc Interpretablity](https://mlsanity.com/paper/1806.07498/) (2018)
- [Scaling up Trustless DNN Inference with Zero-Knowledge Proofs](https://mlsanity.com/paper/2210.08674/) (2022)
- [Active Fairness Auditing](https://mlsanity.com/paper/2206.08450/) (2022)
- [Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes](https://mlsanity.com/paper/1903.08778/) (2019)
- [Looking Deeper into Tabular LIME](https://mlsanity.com/paper/2008.11092/) (2020)
- [pvCNN: Privacy-Preserving and Verifiable Convolutional Neural Network Testing](https://mlsanity.com/paper/2201.09186/) (2022)
- [On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness](https://mlsanity.com/paper/2108.05379/) (2021)
- [Fairness, Integrity, and Privacy in a Scalable Blockchain-based Federated Learning System](https://mlsanity.com/paper/2111.06290/) (2021)
- [Trustless Audits without Revealing Data or Models](https://mlsanity.com/paper/2404.04500/) (2024)
- [FairProof : Confidential and Certifiable Fairness for Neural Networks](https://mlsanity.com/paper/2402.12572/) (2024)
- [Auditing Local Explanations is Hard](https://mlsanity.com/paper/2407.13281/) (2024)
- [Verifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems](https://mlsanity.com/paper/2309.06061/) (2023)
- [XAudit : A Theoretical Look at Auditing with Explanations](https://mlsanity.com/paper/2206.04740/) (2022)
- [Influence-based Attributions can be Manipulated](https://mlsanity.com/paper/2409.05208/) (2024)

## Built on by (most cited first, 2 of 2)

- [QML-PipeGuard: Drift-Aware Behavioral Fingerprinting for Quantum Machine Learning Pipeline Integrity](https://mlsanity.com/paper/2605.25066/) (2026)
- [Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference](https://mlsanity.com/paper/2609.36437/) (2026)

## Abstract (arXiv)

In principle, explanations are intended as a way to increase trust in machine learning models and are often obligated by regulations. However, many circumstances where these are demanded are adversarial in nature, meaning the involved parties have misaligned interests and are incentivized to manipulate explanations for their purpose. As a result, explainability methods fail to be operational in such settings despite the demand citebordt2022post. In this paper, we take a step towards operationalizing explanations in adversarial scenarios with Zero-Knowledge Proofs (ZKPs), a cryptographic primitive. Specifically we explore ZKP-amenable versions of the popular explainability algorithm LIME and evaluate their performance on Neural Networks and Random Forests. Our code is publicly available at https://github.com/emlaufer/ExpProof.