# Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Features

arXiv: https://arxiv.org/abs/2104.00411 · HTML: https://mlsanity.com/paper/2104.00411/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [Deep Learning COVID-19 Features on CXR using Limited Training Data Sets](https://mlsanity.com/paper/2004.05758/) (2020)
- [A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations](https://mlsanity.com/paper/1805.07039/) (2018)
- [Thoracic Disease Identification and Localization with Limited Supervision](https://mlsanity.com/paper/1711.06373/) (2017)
- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [InfoMask: Masked Variational Latent Representation to Localize Chest Disease](https://mlsanity.com/paper/1903.11741/) (2019)
- [Automated diagnosis of COVID-19 with limited posteroanterior chest X-ray images using fine-tuned deep neural networks](https://mlsanity.com/paper/2004.11676/) (2020)
- [DeepCOVIDExplainer: Explainable COVID-19 Diagnosis Based on Chest X-ray Images](https://mlsanity.com/paper/2004.04582/) (2020)
- [Learning Interpretable Features via Adversarially Robust Optimization](https://mlsanity.com/paper/1905.03767/) (2019)
- [BS-Net: learning COVID-19 pneumonia severity on a large Chest X-Ray dataset](https://mlsanity.com/paper/2006.04603/) (2020)
- [Rethinking Positive Aggregation and Propagation of Gradients in Gradient-based Saliency Methods](https://mlsanity.com/paper/2012.00362/) (2020)

## Built on by (most cited first, 9 of 9)

- [CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN](https://mlsanity.com/paper/2207.07553/) (2022)
- [Do Explanations Explain? Model Knows Best](https://mlsanity.com/paper/2203.02269/) (2022)
- [Inherently Interpretable Multi-Label Classification Using Class-Specific Counterfactuals](https://mlsanity.com/paper/2303.00500/) (2023)
- [Towards Semantic Interpretation of Thoracic Disease and COVID-19 Diagnosis Models](https://mlsanity.com/paper/2104.02481/) (2021)
- [Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images](https://mlsanity.com/paper/2303.08632/) (2023)
- [Interpretable Vertebral Fracture Diagnosis](https://mlsanity.com/paper/2203.16273/) (2022)
- [A clinically motivated self-supervised approach for content-based image retrieval of CT liver images](https://mlsanity.com/paper/2207.04812/) (2022)
- [A Dual-Perspective Approach to Evaluating Feature Attribution Methods](https://mlsanity.com/paper/2308.08949/) (2023)
- [Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models](https://mlsanity.com/paper/2204.01729/) (2022)

## Abstract (arXiv)

Neural networks have demonstrated remarkable performance in classification and regression tasks on chest X-rays. In order to establish trust in the clinical routine, the networks' prediction mechanism needs to be interpretable. One principal approach to interpretation is feature attribution. Feature attribution methods identify the importance of input features for the output prediction. Building on Information Bottleneck Attribution (IBA) method, for each prediction we identify the chest X-ray regions that have high mutual information with the network's output. Original IBA identifies input regions that have sufficient predictive information. We propose Inverse IBA to identify all informative regions. Thus all predictive cues for pathologies are highlighted on the X-rays, a desirable property for chest X-ray diagnosis. Moreover, we propose Regression IBA for explaining regression models. Using Regression IBA we observe that a model trained on cumulative severity score labels implicitly learns the severity of different X-ray regions. Finally, we propose Multi-layer IBA to generate higher resolution and more detailed attribution/saliency maps. We evaluate our methods using both human-centric (ground-truth-based) interpretability metrics, and human-independent feature importance metrics on NIH Chest X-ray8 and BrixIA datasets. The Code is publicly available.