Medical Report Generation Is A Multi-label Classification Problem
Not on the map. Why?
Abstract (arXiv)
Medical report generation is a critical task in healthcare that involves the automatic creation of detailed and accurate descriptions from medical images. Traditionally, this task has been approached as a sequence generation problem, relying on vision-and-language techniques to generate coherent and contextually relevant reports. However, in this paper, we propose a novel perspective: rethinking medical report generation as a multi-label classification problem. By framing the task this way, we leverage the radiology nodes from the commonly used knowledge graph, which can be better captured through classification techniques. To verify our argument, we introduce a novel report generation framework based on BLIP integrated with classified key nodes, which allows for effective report generation with accurate classification of multiple key aspects within the medical images. This approach not only simplifies the report generation process but also significantly enhances performance metrics. Our extensive experiments demonstrate that leveraging key nodes can achieve state-of-the-art (SOTA) performance, surpassing existing approaches across two benchmark datasets. The results underscore the potential of re-envisioning traditional tasks with innovative methodologies, paving the way for more efficient and accurate medical report generation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 30 Aug 2024.
Builds on
30 of 41 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2020Generating Radiology Reports via Memory-driven Transformercited by 363
- 2019Meshed-Memory Transformer for Image Captioningcited by 343
18 more
- 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
- 2017On the Automatic Generation of Medical Imaging Reportscited by 215
- 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
- 2020When Radiology Report Generation Meets Knowledge Graphcited by 144
- 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
- 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
- 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
- 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 111
- 2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 98
- 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 92
- 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
- 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 60
- 2022A Medical Semantic-Assisted Transformer for Radiographic Report Generationcited by 42
- 2020Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generationcited by 35
- 2020A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Imagescited by 35
- 2022Cross-modal Clinical Graph Transformer for Ophthalmic Report Generationcited by 24
- 2023Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretrainingcited by 11
- 2024Contrastive Learning with Counterfactual Explanations for Radiology Report Generationcited by 8
Built on by
Citing papers per year: 2026: 2.
2 papers.