mlsanityData to 14 Oct 2026

KiUT: Knowledge-injected U-Transformer for Radiology Report Generation

arXiv 2306.11345 ↗First posted 20 Jun 2023cs.CV cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

Radiology report generation aims to automatically generate a clinically accurate and coherent paragraph from the X-ray image, which could relieve radiologists from the heavy burden of report writing. Although various image caption methods have shown remarkable performance in the natural image field, generating accurate reports for medical images requires knowledge of multiple modalities, including vision, language, and medical terminology. We propose a Knowledge-injected U-Transformer (KiUT) to learn multi-level visual representation and adaptively distill the information with contextual and clinical knowledge for word prediction. In detail, a U-connection schema between the encoder and decoder is designed to model interactions between different modalities. And a symptom graph and an injected knowledge distiller are developed to assist the report generation. Experimentally, we outperform state-of-the-art methods on two widely used benchmark datasets: IU-Xray and MIMIC-CXR. Further experimental results prove the advantages of our architecture and the complementary benefits of the injected knowledge.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

36 of 46 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  11. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  12. 2016Self-critical Sequence Training for Image Captioningcited by 547
24 more
  1. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  2. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  3. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 232
  4. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  5. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
  6. 2019Image Captioning: Transforming Objects into Wordscited by 171
  7. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 162
  8. 2020When Radiology Report Generation Meets Knowledge Graphcited by 145
  9. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  10. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  11. 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
  12. 2016Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challengecited by 116
  13. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  14. 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
  15. 2022AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generationcited by 65
  16. 2020Normalized and Geometry-Aware Self-Attention Network for Image Captioningcited by 63
  17. 2019Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichmentcited by 60
  18. 2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 59
  19. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  20. 2020Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Networkcited by 48
  21. 2020Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generationcited by 35
  22. 2021RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reportingcited by 34
  23. 2017Recurrent Topic-Transition GAN for Visual Paragraph Generationcited by 25
  24. 2022Cross-modal Clinical Graph Transformer for Ophthalmic Report Generationcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 23; 2025: 19; 2026: 13.

62 papers.

Sort:

By area:

  1. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 114
  2. 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 95
  3. 2024Automated Radiology Report Generation: A Review of Recent Advancescited by 43
  4. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 42
  5. 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 35
  6. 2024HERGen: Elevating Radiology Report Generation with Longitudinal Datacited by 33
  7. 2023RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoningcited by 22
  8. 2023Complex Organ Mask Guided Radiology Report Generationcited by 22
  9. 2023LLM-driven Multimodal Target Volume Contouring in Radiation Oncologycited by 20
  10. 2024Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlookscited by 18
  11. 2023A Systematic Review of Deep Learning-based Research on Radiology Report Generationcited by 14
  12. 2025Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generationcited by 13
38 more
  1. 2024R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generationcited by 12
  2. 2024Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generationcited by 11
  3. 2024MedRAT: Unpaired Medical Report Generation via Auxiliary Taskscited by 9
  4. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  5. 2024Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Modelscited by 8
  6. 2025DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generationcited by 8
  7. 2025CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generationcited by 8
  8. 2024Pre-training on High Definition X-ray Images: An Experimental Studycited by 7
  9. 2025A Chain of Diagnosis Framework for Accurate and Explainable Radiology Report Generationcited by 7
  10. 2025Med-GLIP: Advancing Medical Language-Image Pre-training with Large-scale Grounded Datasetcited by 7
  11. 2024Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLMcited by 6
  12. 2024A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Datacited by 6
  13. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generationcited by 5
  14. 2024EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledgecited by 4
  15. 2024MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancementcited by 4
  16. 2025LLM-driven Medical Report Generation via Communication-efficient Heterogeneous Federated Learningcited by 4
  17. 2024TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generationcited by 3
  18. 2024Image-aware Evaluation of Generated Medical Reportscited by 3
  19. 2024ORID: Organ-Regional Information Driven Framework for Radiology Report Generationcited by 3
  20. 2025MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generationcited by 3
  21. 2025EMRRG: Efficient Fine-Tuning Pre-trained X-ray Mamba Networks for Radiology Report Generationcited by 3
  22. 2025Generative AI for Healthcare: Fundamentals, Challenges, and Perspectivescited by 3
  23. 2024ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentationcited by 2
  24. 2024MedCycle: Unpaired Medical Report Generation via Cycle-Consistencycited by 2
  25. 2024Medical Report Generation Is A Multi-label Classification Problemcited by 2
  26. 2024FODA-PG for Enhanced Medical Imaging Narrative Generation: Adaptive Differentiation of Normal and Abnormal Attributescited by 2
  27. 2025Vision-Language Models for Automated 3D PET/CT Report Generationcited by 2
  28. 2024Dynamic Traceback Learning for Medical Report Generationcited by 1
  29. 2024FITA: Fine-grained Image-Text Aligner for Radiology Report Generationcited by 1
  30. 2024GCS-M3VLT: Guided Context Self-Attention based Multi-modal Medical Vision Language Transformer for Retinal Image Captioningcited by 1
  31. 2025Med-R2: Perception and Reflection-driven Complex Reasoning for Medical Report Generationcited by 1
  32. 2025CLARIFID: Improving Radiology Report Generation by Reinforcing Clinically Accurate Impressions and Enforcing Detailed Findingscited by 1
  33. 2025OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report Generationcited by 1
  34. 2026Bridging the Skill Gap in Clinical CBCT Interpretation with CBCTRepDcited by 1
  35. 2026Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generationcited by 1
  36. 2026RIHA: Report-Image Hierarchical Alignment for Radiology Report Generationcited by 1
  37. 2026Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generationcited by 1
  38. 2023RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistancecited by 0

50 most cited of 62 All 62 (JSON)