# Comparing radiologists' gaze and saliency maps generated by interpretability methods for chest x-rays

arXiv: https://arxiv.org/abs/2112.11716 · HTML: https://mlsanity.com/paper/2112.11716/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound](https://mlsanity.com/paper/1612.05601/) (2016)
- [Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development](https://mlsanity.com/paper/2009.07386/) (2020)
- [REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays](https://mlsanity.com/paper/2109.14187/) (2021)
- [A Locally Weighted Fixation Density-Based Metric for Assessing the Quality of Visual Saliency Predictions](https://mlsanity.com/paper/1708.00169/) (2017)

## Built on by (most cited first, 7 of 7)

- [Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type](https://mlsanity.com/paper/2307.13345/) (2023)
- [Interpretable Prediction of Pulmonary Hypertension in Newborns using Echocardiograms](https://mlsanity.com/paper/2203.13038/) (2022)
- [VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Models](https://mlsanity.com/paper/2410.04609/) (2024)
- [Exploring How Anomalous Model Input and Output Alerts Affect Decision-Making in Healthcare](https://mlsanity.com/paper/2204.13194/) (2022)
- [Training Better Deep Learning Models Using Human Saliency](https://mlsanity.com/paper/2410.16190/) (2024)
- [Saliency Guided Longitudinal Medical Visual Question Answering](https://mlsanity.com/paper/2509.25374/) (2025)
- [Attention Consistent Longitudinal Medical Visual Question Answering Guided by Vision Foundation Models](https://mlsanity.com/paper/2606.06534/) (2026)

## Abstract (arXiv)

The interpretability of medical image analysis models is considered a key research field. We use a dataset of eye-tracking data from five radiologists to compare the outputs of interpretability methods and the heatmaps representing where radiologists looked. We conduct a class-independent analysis of the saliency maps generated by two methods selected from the literature: Grad-CAM and attention maps from an attention-gated model. For the comparison, we use shuffled metrics, which avoid biases from fixation locations. We achieve scores comparable to an interobserver baseline in one shuffled metric, highlighting the potential of saliency maps from Grad-CAM to mimic a radiologist's attention over an image. We also divide the dataset into subsets to evaluate in which cases similarities are higher.