# Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Images

arXiv: https://arxiv.org/abs/2503.00657 · HTML: https://mlsanity.com/paper/2503.00657/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 73 references that are in the map.
- First posted on arXiv 1 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations](https://mlsanity.com/paper/2103.10663/) (2021)
- [Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification](https://mlsanity.com/paper/1803.02315/) (2018)
- [Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development](https://mlsanity.com/paper/2009.07386/) (2020)
- [Thoracic Disease Identification and Localization with Limited Supervision](https://mlsanity.com/paper/1711.06373/) (2017)
- [Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation](https://mlsanity.com/paper/2109.12242/) (2021)
- [Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition](https://mlsanity.com/paper/1312.7570/) (2013)
- [ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classification](https://mlsanity.com/paper/2111.12918/) (2021)
- [REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays](https://mlsanity.com/paper/2109.14187/) (2021)
- [Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis](https://mlsanity.com/paper/2112.12349/) (2021)
- [ImageGCN: Multi-Relational Image Graph Convolutional Networks for Disease Identification with Chest X-rays](https://mlsanity.com/paper/1904.00325/) (2019)
- [Interpretability-Driven Sample Selection Using Self Supervised Learning For Disease Classification And Segmentation](https://mlsanity.com/paper/2104.06087/) (2021)
- [Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays](https://mlsanity.com/paper/1807.06067/) (2018)
- [AnaXNet: Anatomy Aware Multi-label Finding Classification in Chest X-ray](https://mlsanity.com/paper/2105.09937/) (2021)
- [Anatomy-XNet: An Anatomy Aware Convolutional Neural Network for Thoracic Disease Classification in Chest X-rays](https://mlsanity.com/paper/2106.05915/) (2021)
- [Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays](https://mlsanity.com/paper/2207.04394/) (2022)
- [Consistency-Based Semi-supervised Evidential Active Learning for Diagnostic Radiograph Classification](https://mlsanity.com/paper/2209.01858/) (2022)
- [A Generic Approach to Lung Field Segmentation from Chest Radiographs using Deep Space and Shape Learning](https://mlsanity.com/paper/1807.04339/) (2018)

## Abstract (arXiv)

Expert radiologists visually scan Chest X-Ray (CXR) images, sequentially fixating on anatomical structures to perform disease diagnosis. An automatic multi-label classifier of diseases in CXR images can benefit by incorporating aspects of the radiologists' approach. Recorded visual scanpaths of radiologists on CXR images can be used for the said purpose. But, such scanpaths are not available for most CXR images, which creates a gap even for modern deep learning based classifiers. This paper proposes to mitigate this gap by generating effective artificial visual scanpaths using a visual scanpath prediction model for CXR images. Further, a multi-class multi-label classifier framework is proposed that uses a generated scanpath and visual image features to classify diseases in CXR images. While the scanpath predictor is based on a recurrent neural network, the multi-label classifier involves a novel iterative sequential model with an attention module. We show that our scanpath predictor generates human-like visual scanpaths. We also demonstrate that the use of artificial visual scanpaths improves multi-class multi-label disease classification results on CXR images. The above observations are made from experiments involving around 0.2 million CXR images from 2 widely-used datasets considering the multi-label classification of 14 pathological findings. Code link: https://github.com/ashishverma03/SDC