mlsanityData to 14 Oct 2026

Cross-modal Memory Networks for Radiology Report Generation

arXiv 2204.13258 ↗First posted 28 Apr 2022cs.CLcited by 185

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later treatments. By generating the reports automatically, it is beneficial to help lighten the burden of radiologists and significantly promote clinical automation, which already attracts much attention in applying artificial intelligence to medical domain. Previous studies mainly follow the encoder-decoder paradigm and focus on the aspect of text generation, with few studies considering the importance of cross-modal mappings and explicitly exploit such mappings to facilitate radiology report generation. In this paper, we propose a cross-modal memory networks (CMN) to enhance the encoder-decoder framework for radiology report generation, where a shared memory is designed to record the alignment between images and texts so as to facilitate the interaction and generation across modalities. Experimental results illustrate the effectiveness of our proposed model, where state-of-the-art performance is achieved on two widely used benchmark datasets, i.e., IU X-Ray and MIMIC-CXR. Further analyses also prove that our model is able to better align information from radiology images and texts so as to help generating more accurate reports in terms of clinical indicators.

Key facts. Cited by 185 papers in the map (data through 14 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

28 of 48 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  10. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  11. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  12. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
16 more
  1. 2016Self-critical Sequence Training for Image Captioningcited by 547
  2. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  3. 2014Memory Networkscited by 363
  4. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  5. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 232
  6. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  7. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  8. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  9. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 135
  10. 2018Relational recurrent neural networkscited by 65
  11. 2020Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reportscited by 58
  12. 2018Learning to Summarize Radiology Findingscited by 54
  13. 2018Topic Memory Networks for Short Text Classificationcited by 25
  14. 2021ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoderscited by 22
  15. 2020Improving Named Entity Recognition with Attentive Ensemble of Syntactic Informationcited by 17
  16. 2020Keyphrase Generation with Cross-Document Attentioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 34; 2024: 63; 2025: 40; 2026: 41.

185 papers.

Sort:

By area:

  1. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 140
  2. 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 118
  3. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 114
  4. 2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 101
  5. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 83
  6. 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 62
  7. 2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 59
  8. 2023PRIOR: Prototype Representation Joint Learning from Medical Images and Reportscited by 55
  9. 2022A Medical Semantic-Assisted Transformer for Radiographic Report Generationcited by 44
  10. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  11. 2024Automated Radiology Report Generation: A Review of Recent Advancescited by 43
  12. 2023ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoningcited by 42
38 more
  1. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 42
  2. 2023A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysiscited by 37
  3. 2023Towards Unifying Medical Vision-and-Language Pre-training via Soft Promptscited by 36
  4. 2023Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reportscited by 35
  5. 2024HERGen: Elevating Radiology Report Generation with Longitudinal Datacited by 33
  6. 2024Dia-LLaMA: Towards Large Language Model-driven CT Report Generationcited by 32
  7. 2024MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generationcited by 29
  8. 2024Libra: Leveraging Temporal Images for Biomedical Radiology Analysiscited by 29
  9. 2024Large Language Model with Region-guided Referring and Grounding for CT Report Generationcited by 28
  10. 2023Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generationcited by 24
  11. 2024Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Modelscited by 24
  12. 2023Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generationcited by 23
  13. 2023Controllable Chest X-Ray Report Generation from Longitudinal Representationscited by 23
  14. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  15. 2022CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generationcited by 22
  16. 2023Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-trainingcited by 22
  17. 2023RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoningcited by 22
  18. 2022Attributed Abnormality Graph Embedding for Clinically Accurate X-Ray Report Generationcited by 17
  19. 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancementcited by 16
  20. 20243D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Modelscited by 16
  21. 2024Radiology Report Generation via Multi-objective Preference Optimizationcited by 16
  22. 2023Cross-Modal Causal Intervention for Medical Report Generationcited by 15
  23. 2024Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generationcited by 15
  24. 2023ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Datacited by 14
  25. 2023Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognitioncited by 14
  26. 2023A Systematic Review of Deep Learning-based Research on Radiology Report Generationcited by 14
  27. 2023A scoping review on multimodal deep learning in biomedical images and textscited by 13
  28. 2023Improving Image Captioning via Predicting Structured Conceptscited by 13
  29. 2025RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injectioncited by 13
  30. 2024R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generationcited by 12
  31. 2025RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignmentcited by 12
  32. 2024Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generationcited by 11
  33. 2024Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generationcited by 11
  34. 2024A Survey of Medical Vision-and-Language Applications and Their Techniquescited by 11
  35. 2023Token Imbalance Adaptation for Radiology Report Generationcited by 10
  36. 2023Customizing General-Purpose Foundation Models for Medical Report Generationcited by 9
  37. 2024Voice Attribute Editing with Text Promptcited by 9
  38. 2024MedRAT: Unpaired Medical Report Generation via Auxiliary Taskscited by 9

50 most cited of 185 All 185 (JSON)