mlsanityData to 14 Oct 2026

Competence-based Multimodal Curriculum Learning for Medical Report Generation

arXiv 2206.14579 ↗First posted 24 Jun 2022cs.CL cs.CV cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

Medical report generation task, which targets to produce long and coherent descriptions of medical images, has attracted growing research interests recently. Different from the general image captioning tasks, medical report generation is more challenging for data-driven neural models. This is mainly due to 1) the serious data bias and 2) the limited medical data. To alleviate the data bias and make best use of available data, we propose a Competence-based Multimodal Curriculum Learning framework (CMCL). Specifically, CMCL simulates the learning process of radiologists and optimizes the model in a step by step manner. Firstly, CMCL estimates the difficulty of each training instance and evaluates the competence of current model; Secondly, CMCL selects the most suitable batch of training instances considering current model competence. By iterating above two steps, CMCL can gradually improve the model's performance. The experiments on the public IU-Xray and MIMIC-CXR datasets show that CMCL can be incorporated into existing models to improve their performance.

Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). Builds on 45 of its 63 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

45 of 63 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  7. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  8. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  9. 2016Self-critical Sequence Training for Image Captioningcited by 547
  10. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  11. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  12. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 232
33 more
  1. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  2. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 202
  3. 2020X-Linear Attention Networks for Image Captioningcited by 198
  4. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 162
  5. 2020When Radiology Report Generation Meets Knowledge Graphcited by 145
  6. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  7. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  8. 2018TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rayscited by 139
  9. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  10. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
  11. 2016A Hierarchical Approach for Generating Descriptive Image Paragraphscited by 104
  12. 2019Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generationcited by 102
  13. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  14. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  15. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  16. 2022AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generationcited by 65
  17. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  18. 2019Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichmentcited by 60
  19. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  20. 2016Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotationcited by 59
  21. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  22. 2020Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reportscited by 58
  23. 2015Video Paragraph Captioning Using Hierarchical Recurrent Neural Networkscited by 54
  24. 2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
  25. 2019Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representationscited by 36
  26. 2019Reinforcement Learning based Curriculum Optimization for Neural Machine Translationcited by 29
  27. 2017Recurrent Topic-Transition GAN for Visual Paragraph Generationcited by 25
  28. 2018simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captionscited by 20
  29. 2020Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Modelscited by 20
  30. 2020Exploring and Distilling Cross-Modal Information for Image Captioningcited by 19
  31. 2020A Competence-aware Curriculum for Visual Concepts Learning via Question Answeringcited by 16
  32. 2019Addressing Data Bias Problems for Chest X-ray Image Report Generationcited by 14
  33. 2020Dynamic Curriculum Learning for Low-Resource Neural Machine Translationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 25; 2024: 18; 2025: 14; 2026: 11.

76 papers.

Sort:

By area:

  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  2. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  3. 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 118
  4. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  5. 2024Automated Radiology Report Generation: A Review of Recent Advancescited by 43
  6. 2023ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoningcited by 42
  7. 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 35
  8. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 29
  9. 2023Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generationcited by 23
  10. 2022CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generationcited by 22
  11. 2023RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoningcited by 22
  12. 2023Complex Organ Mask Guided Radiology Report Generationcited by 22
38 more
  1. 2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
  2. 2023Cross-Modal Causal Intervention for Medical Report Generationcited by 15
  3. 2022Aligning Source Visual and Target Language Domains for Unpaired Video Captioningcited by 14
  4. 2023ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Datacited by 14
  5. 2023A Systematic Review of Deep Learning-based Research on Radiology Report Generationcited by 14
  6. 2023A scoping review on multimodal deep learning in biomedical images and textscited by 13
  7. 2024Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generationcited by 11
  8. 2024A Survey of Medical Vision-and-Language Applications and Their Techniquescited by 11
  9. 2022Prophet Attention: Predicting Attention with Future Attention for Image Captioningcited by 10
  10. 2023Token Imbalance Adaptation for Radiology Report Generationcited by 10
  11. 2025DART: Disease-aware Image-Text Alignment and Self-correcting Re-alignment for Trustworthy Radiology Report Generationcited by 9
  12. 2024Contrastive Learning with Counterfactual Explanations for Radiology Report Generationcited by 8
  13. 2024Pre-training on High Definition X-ray Images: An Experimental Studycited by 7
  14. 2024A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Datacited by 6
  15. 2024A Psychology-based Unified Dynamic Framework for Curriculum Learningcited by 6
  16. 2023Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmarkcited by 5
  17. 2025Multimodal Large Language Models for Medical Report Generation via Customized Prompt Tuningcited by 5
  18. 2025A Disease-Aware Dual-Stage Framework for Chest X-ray Report Generationcited by 5
  19. 2022Improving Radiology Summarization with Radiograph and Anatomy Promptscited by 4
  20. 2022In-sample Curriculum Learning by Sequence Completion for Natural Language Generationcited by 4
  21. 2023The Role of Data Curation in Image Captioningcited by 4
  22. 2023Two-Stage Triplet Loss Training with Curriculum Augmentation for Audio-Visual Retrievalcited by 4
  23. 2025Walking the Tightrope: Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuningcited by 4
  24. 2025R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generationcited by 4
  25. 2022Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penaltycited by 3
  26. 2022Cross-Modal Similarity-Based Curriculum Learning for Image Captioningcited by 3
  27. 2023CellMix: A General Instance Relationship based Method for Data Augmentation Towards Pathology Image Classificationcited by 3
  28. 2023Complexity-Guided Curriculum Learning for Text Graphscited by 3
  29. 2023Medical Report Generation based on Segment-Enhanced Contrastive Representation Learningcited by 3
  30. 2024Exploring Curriculum Learning for Vision-Language Tasks: A Study on Small-Scale Multimodal Trainingcited by 3
  31. 2025GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generationcited by 3
  32. 2023Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledgecited by 2
  33. 2023Curriculum Learning for Graph Neural Networks: A Multiview Competence-based Approachcited by 2
  34. 2023Can Prompt Learning Benefit Radiology Report Generation?cited by 2
  35. 2024Data Distribution-based Curriculum Learningcited by 2
  36. 2024ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentationcited by 2
  37. 2024WoLF: Wide-scope Large Language Model Framework for CXR Understandingcited by 2
  38. 2024See Detail Say Clear: Towards Brain CT Report Generation via Pathological Clue-driven Representation Learningcited by 2

50 most cited of 76 All 76 (JSON)