The Definitions of Interpretability and Learning of Interpretable Models
Not on the map. Why?
Abstract (arXiv)
As machine learning algorithms getting adopted in an ever-increasing number of applications, interpretation has emerged as a crucial desideratum. In this paper, we propose a mathematical definition for the human-interpretable model. In particular, we define interpretability between two information process systems. If a prediction model is interpretable by a human recognition system based on the above interpretability definition, the prediction model is defined as a completely human-interpretable model. We further design a practical framework to train a completely human-interpretable model by user interactions. Experiments on image datasets show the advantages of our proposed model in two aspects: 1) The completely human-interpretable model can provide an entire decision-making process that is human-understandable; 2) The completely human-interpretable model is more robust against adversarial attacks.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 29 May 2021.
Builds on
19 of 31 references.
References · 19
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2020Concept Bottleneck Modelscited by 1,032
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
- 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
- 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
7 more
- 2020Concept Whitening for Interpretable Image Recognitioncited by 225
- 2020Compositional Explanations of Neuronscited by 172
- 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
- 2017SafetyNet: Detecting and Rejecting Adversarial Examples Robustlycited by 76
- 2019Interpretability Beyond Classification Output: Semantic Bottleneck Networkscited by 49
- 2020Proper Network Interpretability Helps Adversarial Robustness in Classificationcited by 41
- 2018Semantic bottleneck for computer vision taskscited by 10
Built on by
Citing papers per year: 2021: 1.
1 paper.