mlsanityData to 7 Oct 2026

Medical Report Generation Is A Multi-label Classification Problem

arXiv 2409.00250 ↗First posted 30 Aug 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Medical report generation is a critical task in healthcare that involves the automatic creation of detailed and accurate descriptions from medical images. Traditionally, this task has been approached as a sequence generation problem, relying on vision-and-language techniques to generate coherent and contextually relevant reports. However, in this paper, we propose a novel perspective: rethinking medical report generation as a multi-label classification problem. By framing the task this way, we leverage the radiology nodes from the commonly used knowledge graph, which can be better captured through classification techniques. To verify our argument, we introduce a novel report generation framework based on BLIP integrated with classified key nodes, which allows for effective report generation with accurate classification of multiple key aspects within the medical images. This approach not only simplifies the report generation process but also significantly enhances performance metrics. Our extensive experiments demonstrate that leveraging key nodes can achieve state-of-the-art (SOTA) performance, surpassing existing approaches across two benchmark datasets. The results underscore the potential of re-envisioning traditional tasks with innovative methodologies, paving the way for more efficient and accurate medical report generation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 30 Aug 2024.

Builds on

30 of 41 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  9. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  11. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  12. 2019Meshed-Memory Transformer for Image Captioningcited by 343
18 more
  1. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  2. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  3. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
  4. 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
  5. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  6. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  7. 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
  8. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 111
  9. 2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 98
  10. 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 92
  11. 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
  12. 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 60
  13. 2022A Medical Semantic-Assisted Transformer for Radiographic Report Generationcited by 42
  14. 2020Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generationcited by 35
  15. 2020A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Imagescited by 35
  16. 2022Cross-modal Clinical Graph Transformer for Ophthalmic Report Generationcited by 24
  17. 2023Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretrainingcited by 11
  18. 2024Contrastive Learning with Counterfactual Explanations for Radiology Report Generationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.