mlsanityData to 14 Oct 2026

Interpretability Guarantees with Merlin-Arthur Classifiers

arXiv 2206.00759 ↗First posted 1 Jun 2022cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose an interactive multi-agent classifier that provides provable interpretability guarantees even for complex agents such as neural networks. These guarantees consist of lower bounds on the mutual information between selected features and the classification decision. Our results are inspired by the Merlin-Arthur protocol from Interactive Proof Systems and express these bounds in terms of measurable metrics such as soundness and completeness. Compared to existing interactive setups, we rely neither on optimal agents nor on the assumption that features are distributed independently. Instead, we use the relative strength of the agents as well as the new concept of Asymmetric Feature Correlation which captures the precise kind of correlations that make interpretability guarantees difficult. We evaluate our results on two small-scale datasets where high mutual information can be verified explicitly.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 39 of its 63 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

39 of 63 references.

References · 39

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  6. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  7. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  8. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  9. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  10. 2018AI safety via debatecited by 400
  11. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  12. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
27 more
  1. 2016Rationalizing Neural Predictionscited by 281
  2. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  3. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  4. 2019Explanations can be manipulated and geometry is to blamecited by 216
  5. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 208
  6. 2018Abduction-Based Explanations for Machine Learning Modelscited by 160
  7. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  8. 2018A Symbolic Approach to Explaining Bayesian Network Classifierscited by 131
  9. 2020Invariant Rationalizationcited by 131
  10. 2019Interpretable Neural Predictions with Differentiable Binary Variablescited by 126
  11. 2017Stabilizing Training of Generative Adversarial Networks through Regularizationcited by 118
  12. 2021Counterfactual Explanations Can Be Manipulatedcited by 113
  13. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  14. 2019Rethinking Cooperative Rationalization: Introspective Extraction and Complement Controlcited by 96
  15. 2020Shapley explainability on the data manifoldcited by 88
  16. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 72
  17. 2019Stabilizing Generative Adversarial Networks: A Surveycited by 57
  18. 2020On Explaining Decision Treescited by 49
  19. 2019Generative Counterfactual Introspection for Explainable Deep Learningcited by 36
  20. 2019A Game Theoretic Approach to Class-wise Selective Rationalizationcited by 32
  21. 2021Learning to Give Checkable Answers with Prover-Verifier Gamescited by 32
  22. 2021Provably efficient, succinct, and precise explanationscited by 32
  23. 2019A Rate-Distortion Framework for Explaining Neural Network Decisionscited by 24
  24. 2020Deep Neural Network Training with Frank-Wolfecited by 23
  25. 2021Interpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderingscited by 5
  26. 2022Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Fourcited by 3
  27. 2021A Complete Characterisation of ReLU-Invariant Distributionscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 3; 2026: 3.

9 papers.

  1. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  2. 2025Object Centric Concept Bottleneckscited by 10
  3. 2024Models That Prove Their Own Correctnesscited by 7
  4. 2026How to Avoid Debate: Scalable AI Safety via Doubly-Efficient Interactive Proofscited by 2
  5. 2023Hardness of Deceptive Certificate Selectioncited by 0
  6. 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodingscited by 0
  7. 2025Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Modelscited by 0
  8. 2026Learn from A Rationalist: Distilling Intermediate Interpretable Rationalescited by 0
  9. 2026On the Monitorability of Natural Language Autoencoders: Identification vs Discriminationcited by 0