FODA-PG for Enhanced Medical Imaging Narrative Generation: Adaptive Differentiation of Normal and Abnormal Attributes
Not on the map. Why?
Abstract (arXiv)
Automatic Medical Imaging Narrative generation aims to alleviate the workload of radiologists by producing accurate clinical descriptions directly from radiological images. However, the subtle visual nuances and domain-specific terminology in medical images pose significant challenges compared to generic image captioning tasks. Existing approaches often neglect the vital distinction between normal and abnormal findings, leading to suboptimal performance. In this work, we propose FODA-PG, a novel Fine-grained Organ-Disease Adaptive Partitioning Graph framework that addresses these limitations through domain-adaptive learning. FODA-PG constructs a granular graphical representation of radiological findings by separating disease-related attributes into distinct "disease-specific" and "disease-free" categories based on their clinical significance and location. This adaptive partitioning enables our model to capture the nuanced differences between normal and pathological states, mitigating the impact of data biases. By integrating this fine-grained semantic knowledge into a powerful transformer-based architecture and providing rigorous mathematical justifications for its effectiveness, FODA-PG generates precise and clinically coherent reports with enhanced generalization capabilities. Extensive experiments on the IU-Xray and MIMIC-CXR benchmarks demonstrate the superiority of our approach over state-of-the-art methods, highlighting the importance of domain adaptation in medical report generation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 6 Sep 2024.
Builds on
47 of 60 references.
References · 47
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
35 more
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
- 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
- 2014Show and Tell: A Neural Image Caption Generatorcited by 877
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
- 2020Generating Radiology Reports via Memory-driven Transformercited by 363
- 2019Meshed-Memory Transformer for Image Captioningcited by 343
- 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
- 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
- 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
- 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 231
- 2017On the Automatic Generation of Medical Imaging Reportscited by 215
- 2022Cross-modal Memory Networks for Radiology Report Generationcited by 183
- 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
- 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
- 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
- 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
- 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
- 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
- 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
- 2019Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichmentcited by 60
- 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 60
- 2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
- 2022Cross-modal Clinical Graph Transformer for Ophthalmic Report Generationcited by 24
- 2023Transforming Visual Scene Graphs to Image Captionscited by 15
- 2020Image Captioning with Context-Aware Auxiliary Guidancecited by 10
- 2023HAAV: Hierarchical Aggregation of Augmented Views for Image Captioningcited by 10
- 2023Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Modelcited by 10
- 2023Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graphcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.