# Anatomy-XNet: An Anatomy Aware Convolutional Neural Network for Thoracic Disease Classification in Chest X-rays

arXiv: https://arxiv.org/abs/2106.05915 · HTML: https://mlsanity.com/paper/2106.05915/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [CheXclusion: Fairness gaps in deep chest X-ray classifiers](https://mlsanity.com/paper/2003.00827/) (2020)
- [When Radiology Report Generation Meets Knowledge Graph](https://mlsanity.com/paper/2002.08277/) (2020)
- [MultiGrain: a unified image embedding for classes and instances](https://mlsanity.com/paper/1902.05509/) (2019)
- [Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labels](https://mlsanity.com/paper/2005.12734/) (2020)
- [Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs](https://mlsanity.com/paper/1807.07532/) (2018)
- [Deep Mining External Imperfect Data for Chest X-ray Disease Screening](https://mlsanity.com/paper/2006.03796/) (2020)
- [Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays](https://mlsanity.com/paper/1807.06067/) (2018)
- [SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images](https://mlsanity.com/paper/1810.12959/) (2018)
- [COVID-19 Classification of X-ray Images Using Deep Neural Networks](https://mlsanity.com/paper/2010.01362/) (2020)
- [Segmentation of Lungs in Chest X-Ray Image Using Generative Adversarial Networks](https://mlsanity.com/paper/2009.05752/) (2020)
- [Revisiting CycleGAN for semi-supervised segmentation](https://mlsanity.com/paper/1908.11569/) (2019)

## Built on by (most cited first, 11 of 11)

- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data](https://mlsanity.com/paper/2309.15529/) (2023)
- [ThoraX-PriorNet: A Novel Attention-Based Architecture Using Anatomical Prior Probability Maps for Thoracic Disease Classification](https://mlsanity.com/paper/2210.02998/) (2022)
- [Deep learning for understanding multilabel imbalanced Chest X-ray datasets](https://mlsanity.com/paper/2207.14408/) (2022)
- [Learning to Generalize towards Unseen Domains via a Content-Aware Style Invariant Model for Disease Detection from Chest X-rays](https://mlsanity.com/paper/2302.13991/) (2023)
- [Category-wise Fine-Tuning: Resisting Incorrect Pseudo-Labels in Multi-Label Image Classification with Partial Labels](https://mlsanity.com/paper/2401.16991/) (2024)
- [Rep-GLS: Report-Guided Generalized Label Smoothing for Robust Disease Detection](https://mlsanity.com/paper/2508.02495/) (2025)
- [MVC: A Multi-Task Vision Transformer Network for COVID-19 Diagnosis from Chest X-ray Images](https://mlsanity.com/paper/2310.00418/) (2023)
- [Improving Pediatric Pneumonia Diagnosis with Adult Chest X-ray Images Utilizing Contrastive Learning and Embedding Similarity](https://mlsanity.com/paper/2404.12958/) (2024)
- [Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Images](https://mlsanity.com/paper/2503.00657/) (2025)
- [Anatomic Feature Fusion Model for Diagnosing Calcified Pulmonary Nodules on Chest X-Ray](https://mlsanity.com/paper/2508.12562/) (2025)

## Abstract (arXiv)

Thoracic disease detection from chest radiographs using deep learning methods has been an active area of research in the last decade. Most previous methods attempt to focus on the diseased organs of the image by identifying spatial regions responsible for significant contributions to the model's prediction. In contrast, expert radiologists first locate the prominent anatomical structures before determining if those regions are anomalous. Therefore, integrating anatomical knowledge within deep learning models could bring substantial improvement in automatic disease classification. Motivated by this, we propose Anatomy-XNet, an anatomy-aware attention-based thoracic disease classification network that prioritizes the spatial features guided by the pre-identified anatomy regions. We adopt a semi-supervised learning method by utilizing available small-scale organ-level annotations to locate the anatomy regions in large-scale datasets where the organ-level annotations are absent. The proposed Anatomy-XNet uses the pre-trained DenseNet-121 as the backbone network with two corresponding structured modules, the Anatomy Aware Attention (A3) and Probabilistic Weighted Average Pooling (PWAP), in a cohesive framework for anatomical attention learning. We experimentally show that our proposed method sets a new state-of-the-art benchmark by achieving an AUC score of 85.78%, 92.07%, and, 84.04% on three publicly available large-scale CXR datasets--NIH, Stanford CheXpert, and MIMIC-CXR, respectively. This not only proves the efficacy of utilizing the anatomy segmentation knowledge to improve the thoracic disease classification but also demonstrates the generalizability of the proposed framework.