mlsanityData to 7 Oct 2026

FODA-PG for Enhanced Medical Imaging Narrative Generation: Adaptive Differentiation of Normal and Abnormal Attributes

arXiv 2409.03947 ↗First posted 6 Sep 2024cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Automatic Medical Imaging Narrative generation aims to alleviate the workload of radiologists by producing accurate clinical descriptions directly from radiological images. However, the subtle visual nuances and domain-specific terminology in medical images pose significant challenges compared to generic image captioning tasks. Existing approaches often neglect the vital distinction between normal and abnormal findings, leading to suboptimal performance. In this work, we propose FODA-PG, a novel Fine-grained Organ-Disease Adaptive Partitioning Graph framework that addresses these limitations through domain-adaptive learning. FODA-PG constructs a granular graphical representation of radiological findings by separating disease-related attributes into distinct "disease-specific" and "disease-free" categories based on their clinical significance and location. This adaptive partitioning enables our model to capture the nuanced differences between normal and pathological states, mitigating the impact of data biases. By integrating this fine-grained semantic knowledge into a powerful transformer-based architecture and providing rigorous mathematical justifications for its effectiveness, FODA-PG generates precise and clinically coherent reports with enhanced generalization capabilities. Extensive experiments on the IU-Xray and MIMIC-CXR benchmarks demonstrate the superiority of our approach over state-of-the-art methods, highlighting the importance of domain adaptation in medical report generation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 6 Sep 2024.

Builds on

47 of 60 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2017Graph Attention Networkscited by 8,854
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
35 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  2. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  3. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  4. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  5. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  6. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  7. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  8. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  9. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  11. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  12. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  13. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  14. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  15. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  16. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  17. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 231
  18. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  19. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 183
  20. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
  21. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  22. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  23. 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
  24. 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
  25. 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
  26. 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
  27. 2019Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichmentcited by 60
  28. 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 60
  29. 2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
  30. 2022Cross-modal Clinical Graph Transformer for Ophthalmic Report Generationcited by 24
  31. 2023Transforming Visual Scene Graphs to Image Captionscited by 15
  32. 2020Image Captioning with Context-Aware Auxiliary Guidancecited by 10
  33. 2023HAAV: Hierarchical Aggregation of Augmented Views for Image Captioningcited by 10
  34. 2023Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Modelcited by 10
  35. 2023Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graphcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.