# Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification

arXiv: https://arxiv.org/abs/2303.01871 · HTML: https://mlsanity.com/paper/2303.01871/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation](https://mlsanity.com/paper/1904.08771/) (2019)
- [Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information](https://mlsanity.com/paper/2110.01471/) (2021)
- [One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images](https://mlsanity.com/paper/2107.05047/) (2021)
- [Artificial Intelligence in Glioma Imaging: Challenges and Advances](https://mlsanity.com/paper/1911.12886/) (2019)

## Built on by (most cited first, 13 of 13)

- [Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation](https://mlsanity.com/paper/2403.08002/) (2024)
- [Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2](https://mlsanity.com/paper/2411.15802/) (2024)
- [German CheXpert Chest X-ray Radiology Report Labeler](https://mlsanity.com/paper/2306.02777/) (2023)
- [Is visual explanation with Grad-CAM more reliable for deeper neural networks? a case study with automatic pneumothorax diagnosis](https://mlsanity.com/paper/2308.15172/) (2023)
- [WindowNet: Learnable Windows for Chest X-ray Classification](https://mlsanity.com/paper/2306.06038/) (2023)
- [Higher Chest X-ray Resolution Improves Classification Performance](https://mlsanity.com/paper/2306.06051/) (2023)
- [VisTA: Vision-Text Alignment Model with Contrastive Learning using Multimodal Data for Evidence-Driven, Reliable, and Explainable Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2502.01535/) (2025)
- [Automated Labeling of German Chest X-Ray Radiology Reports using Deep Learning](https://mlsanity.com/paper/2306.05997/) (2023)
- [REMEMBER: Retrieval-based Explainable Multimodal Evidence-guided Modeling for Brain Evaluation and Reasoning in Zero- and Few-shot Neurodegenerative Diagnosis](https://mlsanity.com/paper/2504.09354/) (2025)
- [Systematic Evaluation of Attribution Methods: Eliminating Threshold Bias and Revealing Method-Dependent Performance Patterns](https://mlsanity.com/paper/2509.03176/) (2025)
- [xAI-CV: An Overview of Explainable Artificial Intelligence in Computer Vision](https://mlsanity.com/paper/2509.18913/) (2025)
- [Attention Consistent Longitudinal Medical Visual Question Answering Guided by Vision Foundation Models](https://mlsanity.com/paper/2606.06534/) (2026)
- [Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Study](https://mlsanity.com/paper/2609.25978/) (2026)

## Abstract (arXiv)

Purpose: To investigate chest radiograph (CXR) classification performance of vision transformers (ViT) and interpretability of attention-based saliency using the example of pneumothorax classification. Materials and Methods: In this retrospective study, ViTs were fine-tuned for lung disease classification using four public data sets: CheXpert, Chest X-Ray 14, MIMIC CXR, and VinBigData. Saliency maps were generated using transformer multimodal explainability and gradient-weighted class activation mapping (GradCAM). Classification performance was evaluated on the Chest X-Ray 14, VinBigData, and SIIM-ACR data sets using the area under the receiver operating characteristic curve analysis (AUC) and compared with convolutional neural networks (CNNs). The explainability methods were evaluated with positive/negative perturbation, sensitivity-n, effective heat ratio, intra-architecture repeatability and interarchitecture reproducibility. In the user study, three radiologists classified 160 CXRs with/without saliency maps for pneumothorax and rated their usefulness. Results: ViTs had comparable CXR classification AUCs compared with state-of-the-art CNNs 0.95 (95% CI: 0.943, 0.950) versus 0.83 (95%, CI 0.826, 0.842) on Chest X-Ray 14, 0.84 (95% CI: 0.769, 0.912) versus 0.83 (95% CI: 0.760, 0.895) on VinBigData, and 0.85 (95% CI: 0.847, 0.861) versus 0.87 (95% CI: 0.868, 0.882) on SIIM ACR. Both saliency map methods unveiled a strong bias toward pneumothorax tubes in the models. Radiologists found 47% of the attention-based saliency maps useful and 39% of GradCAM. The attention-based methods outperformed GradCAM on all metrics. Conclusion: ViTs performed similarly to CNNs in CXR classification, and their attention-based saliency maps were more useful to radiologists and outperformed GradCAM.