# CLIMD: A Curriculum Learning Framework for Imbalanced Multimodal Diagnosis

arXiv: https://arxiv.org/abs/2508.01594 · HTML: https://mlsanity.com/paper/2508.01594/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization](https://mlsanity.com/paper/2502.13146/) (2025)
- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [Artificial Intelligence-Based Methods for Fusion of Electronic Health Records and Imaging Data](https://mlsanity.com/paper/2210.13462/) (2022)
- [DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learning](https://mlsanity.com/paper/2503.06456/) (2025)
- [Dynamic Multimodal Information Bottleneck for Multimodality Classification](https://mlsanity.com/paper/2311.01066/) (2023)
- [A scoping review on multimodal deep learning in biomedical images and texts](https://mlsanity.com/paper/2307.07362/) (2023)
- [Deep Multimodal Guidance for Medical Image Classification](https://mlsanity.com/paper/2203.05683/) (2022)

## Built on by (most cited first, 3 of 3)

- [LiMT: A Multi-task Liver Image Benchmark Dataset](https://mlsanity.com/paper/2511.19889/) (2025)
- [SciLT: Long-tailed Image Classification under Scientific Image Domains](https://mlsanity.com/paper/2604.03687/) (2026)
- [Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition](https://mlsanity.com/paper/2605.21565/) (2026)

## Abstract (arXiv)

Clinicians usually combine information from multiple sources to achieve the most accurate diagnosis, and this has sparked increasing interest in leveraging multimodal deep learning for diagnosis. However, in real clinical scenarios, due to differences in incidence rates, multimodal medical data commonly face the issue of class imbalance, which makes it difficult to adequately learn the features of minority classes. Most existing methods tackle this issue with resampling or loss reweighting, but they are prone to overfitting or underfitting and fail to capture cross-modal interactions. Therefore, we propose a Curriculum Learning framework for Imbalanced Multimodal Diagnosis (CLIMD). Specifically, we first design multimodal curriculum measurer that combines two indicators, intra-modal confidence and inter-modal complementarity, to enable the model to focus on key samples and gradually adapt to complex category distributions. Additionally, a class distribution-guided training scheduler is introduced, which enables the model to progressively adapt to the imbalanced class distribution during training. Extensive experiments on multiple multimodal medical datasets demonstrate that the proposed method outperforms state-of-the-art approaches across various metrics and excels in handling imbalanced multimodal medical data. Furthermore, as a plug-and-play CL framework, CLIMD can be easily integrated into other models, offering a promising path for improving multimodal disease diagnosis accuracy. Code is publicly available at https://github.com/KHan-UJS/CLIMD.