A Structure-Aware Relation Network for Thoracic Diseases Detection and Segmentation
Not on the map. Why?
Abstract (arXiv)
Instance level detection and segmentation of thoracic diseases or abnormalities are crucial for automatic diagnosis in chest X-ray images. Leveraging on constant structure and disease relations extracted from domain knowledge, we propose a structure-aware relation network (SAR-Net) extending Mask R-CNN. The SAR-Net consists of three relation modules: 1. the anatomical structure relation module encoding spatial relations between diseases and anatomical parts. 2. the contextual relation module aggregating clues based on query-key pair of disease RoI and lung fields. 3. the disease relation module propagating co-occurrence and causal relations into disease proposals. Towards making a practical system, we also provide ChestX-Det, a chest X-Ray dataset with instance-level annotations (boxes and masks). ChestX-Det is a subset of the public dataset NIH ChestX-ray14. It contains ~3500 images of 13 common disease categories labeled by three board-certified radiologists. We evaluate our SAR-Net on it and another dataset DR-Private. Experimental results show that it can enhance the strong baseline of Mask R-CNN with significant improvements. The ChestX-Det is released at https://github.com/Deepwise-AILab/ChestX-Det-Dataset.
Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 21 Apr 2021.
Builds on
30 of 39 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2017Mask R-CNNcited by 7,259
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2015SSD: Single Shot MultiBox Detectorcited by 3,125
- 2015Fast R-CNNcited by 2,930
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2016Pyramid Scene Parsing Networkcited by 2,472
- 2015Learning Deep Features for Discriminative Localizationcited by 2,430
18 more
- 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
- 2017Deformable Convolutional Networkscited by 1,481
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
- 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
- 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
- 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
- 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
- 2017Relation Networks for Object Detectioncited by 305
- 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
- 2019Bottom-up Object Detection by Grouping Extreme and Center Pointscited by 156
- 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
- 2019Feature Selective Anchor-Free Module for Single-Shot Object Detectioncited by 139
- 2017Thoracic Disease Identification and Localization with Limited Supervisioncited by 55
- 2018Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classificationcited by 36
- 2018Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rayscited by 21
- 2020Deep Hiearchical Multi-Label Classification Applied to Chest X-Ray Abnormality Taxonomiescited by 9
- 2017Learning what to look in chest X-rays with a recurrent visual attention modelcited by 3
Built on by
Citing papers per year: 2021: 1; 2022: 1; 2023: 6; 2024: 11; 2025: 9; 2026: 5.
33 papers.
Sort:
By area:
- 2025MedRAX: Medical Reasoning Agent for Chest X-raycited by 68
- 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
- 2023AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentationcited by 37
- 2023BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeyscited by 29
- 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generationcited by 17
- 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learningcited by 16
- 2023Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretationcited by 14
- 2024Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervisioncited by 12
- 2024A foundation model for generalizable disease diagnosis in chest X-ray imagescited by 12
- 2023Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-Supervisioncited by 11
- 2024Self-supervised visual learning in the low-data regime: a comparative evaluationcited by 11
- 2024S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentationcited by 11
21 more
- 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-trainingcited by 10
- 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
- 2024DWARF: Disease-weighted network for attention map refinementcited by 5
- 2024The FIX Benchmark: Extracting Features Interpretable to eXpertscited by 5
- 2026A unified multi-task framework enables interpretable chest radiograph analysiscited by 5
- 2021GREN: Graph-Regularized Embedding Network for Weakly-Supervised Disease Localization in X-ray Imagescited by 3
- 2025Foundation X: Integrating Classification, Localization, and Segmentation through Lock-Release Pretraining Strategy for Chest X-ray Analysiscited by 3
- 2025Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identificationcited by 2
- 2025Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Maskscited by 2
- 2023Risk Controlled Image Retrievalcited by 1
- 2024Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classificationcited by 1
- 2024Exploring Compositional Generalization of Multimodal LLMs for Medical Imagingcited by 1
- 2025ACE: Anatomically Consistent Embeddings in Composition and Decompositioncited by 1
- 2025Understanding Dataset Bias in Medical Imaging: A Case Study on Chest X-rayscited by 1
- 2026Uncertainty-Aware Compositional Localization and Placement Assessment of Catheters and Tubes in Chest X-Rayscited by 1
- 2024Foundation Model Makes Clustering A Better Initialization For Cold-Start Active Learningcited by 0
- 2025Automatic Fine-grained Segmentation-assisted Report Generationcited by 0
- 2025AURAD: Anatomy-Pathology Unified Radiology Synthesis with Progressive Representationscited by 0
- 2026MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligencecited by 0
- 2026Example-based Robust Abnormality Detection with Minimal Annotations using Exemplar Med-DETRcited by 0
- 2026Anatomy-Decomposed Chest Computed Tomography (CT) Projections as Scalable Supervision for Bone Suppression in Chest Radiographscited by 0
- 2026Anatomy-Decomposed Chest Computed Tomography (CT) Projections as Scalable Supervision for Bone Suppression in Chest Radiographs21 Sep 2026
- 2026Example-based Robust Abnormality Detection with Minimal Annotations using Exemplar Med-DETR25 Aug 2026
- 2026Uncertainty-Aware Compositional Localization and Placement Assessment of Catheters and Tubes in Chest X-Rays11 Aug 2026
- 2026A unified multi-task framework enables interpretable chest radiograph analysis2 Jun 2026
- 2026MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence28 Mar 2026
- 2025Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks10 Nov 2025
- 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation24 Oct 2025
- 2025AURAD: Anatomy-Pathology Unified Radiology Synthesis with Progressive Representations5 Sep 2025
- 2025Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identification25 Jul 2025
- 2025Automatic Fine-grained Segmentation-assisted Report Generation22 Jul 2025
- 2025Understanding Dataset Bias in Medical Imaging: A Case Study on Chest X-rays10 Jul 2025
- 2025Foundation X: Integrating Classification, Localization, and Segmentation through Lock-Release Pretraining Strategy for Chest X-ray Analysis12 Mar 2025
21 more
- 2025MedRAX: Medical Reasoning Agent for Chest X-ray4 Feb 2025
- 2025ACE: Anatomically Consistent Embeddings in Composition and Decomposition17 Jan 2025
- 2024Exploring Compositional Generalization of Multimodal LLMs for Medical Imaging28 Dec 2024
- 2024A foundation model for generalizable disease diagnosis in chest X-ray images11 Oct 2024
- 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation29 Sep 2024
- 2024The FIX Benchmark: Extracting Features Interpretable to eXperts20 Sep 2024
- 2024S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentation12 Aug 2024
- 2024DWARF: Disease-weighted network for attention map refinement24 Jun 2024
- 2024Self-supervised visual learning in the low-data regime: a comparative evaluation26 Apr 2024
- 2024Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision24 Apr 2024
- 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-training4 Apr 2024
- 2024Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classification26 Mar 2024
- 2024Foundation Model Makes Clustering A Better Initialization For Cold-Start Active Learning4 Feb 2024
- 2023Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretation2 Nov 2023
- 2023BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys16 Oct 2023
- 2023Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-Supervision27 Sep 2023
- 2023AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation7 Aug 2023
- 2023Risk Controlled Image Retrieval14 Jul 2023
- 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learning19 Feb 2023
- 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective27 Sep 2022
- 2021GREN: Graph-Regularized Embedding Network for Weakly-Supervised Disease Localization in X-ray Images14 Jul 2021
- 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generationcited by 17
- 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-trainingcited by 10
- 2024Exploring Compositional Generalization of Multimodal LLMs for Medical Imagingcited by 1
- 2026MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligencecited by 0
- 2026MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence28 Mar 2026
- 2024Exploring Compositional Generalization of Multimodal LLMs for Medical Imaging28 Dec 2024
- 2024MedViLaM: A multimodal large language model with advanced generalizability and explainability for medical data understanding and generation29 Sep 2024
- 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-training4 Apr 2024
- 2025MedRAX: Medical Reasoning Agent for Chest X-raycited by 68
- 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
- 2023AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentationcited by 37
- 2023BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeyscited by 29
- 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learningcited by 16
- 2023Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretationcited by 14
- 2024Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervisioncited by 12
- 2024A foundation model for generalizable disease diagnosis in chest X-ray imagescited by 12
- 2023Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-Supervisioncited by 11
- 2024Self-supervised visual learning in the low-data regime: a comparative evaluationcited by 11
- 2024S-SAM: SVD-based Fine-Tuning of Segment Anything Model for Medical Image Segmentationcited by 11
- 2024DWARF: Disease-weighted network for attention map refinementcited by 5
12 most cited of 28 All 28 (JSON)
- 2026Anatomy-Decomposed Chest Computed Tomography (CT) Projections as Scalable Supervision for Bone Suppression in Chest Radiographs21 Sep 2026
- 2026Example-based Robust Abnormality Detection with Minimal Annotations using Exemplar Med-DETR25 Aug 2026
- 2026Uncertainty-Aware Compositional Localization and Placement Assessment of Catheters and Tubes in Chest X-Rays11 Aug 2026
- 2026A unified multi-task framework enables interpretable chest radiograph analysis2 Jun 2026
- 2025Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks10 Nov 2025
- 2025AURAD: Anatomy-Pathology Unified Radiology Synthesis with Progressive Representations5 Sep 2025
- 2025Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identification25 Jul 2025
- 2025Automatic Fine-grained Segmentation-assisted Report Generation22 Jul 2025
- 2025Understanding Dataset Bias in Medical Imaging: A Case Study on Chest X-rays10 Jul 2025
- 2025Foundation X: Integrating Classification, Localization, and Segmentation through Lock-Release Pretraining Strategy for Chest X-ray Analysis12 Mar 2025
- 2025MedRAX: Medical Reasoning Agent for Chest X-ray4 Feb 2025
- 2025ACE: Anatomically Consistent Embeddings in Composition and Decomposition17 Jan 2025
12 newest of 28 All 28 (JSON)