# Robust Incomplete-Modality Alignment for Ophthalmic Disease Grading and Diagnosis via Labeled Optimal Transport

arXiv: https://arxiv.org/abs/2507.04999 · HTML: https://mlsanity.com/paper/2507.04999/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction](https://mlsanity.com/paper/2306.08330/) (2023)
- [Multi-modal Alignment using Representation Codebook](https://mlsanity.com/paper/2203.00048/) (2022)
- [GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges](https://mlsanity.com/paper/2202.06511/) (2022)
- [Tuning Multi-mode Token-level Prompt Alignment across Modalities](https://mlsanity.com/paper/2309.13847/) (2023)
- [Examining Modality Incongruity in Multimodal Federated Learning for Medical Vision and Language-based Disease Detection](https://mlsanity.com/paper/2402.05294/) (2024)
- [Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis](https://mlsanity.com/paper/2502.11724/) (2025)
- [Confidence-aware multi-modality learning for eye disease screening](https://mlsanity.com/paper/2405.18167/) (2024)
- [Cross-modality Matching and Prediction of Perturbation Responses with Labeled Gromov-Wasserstein Optimal Transport](https://mlsanity.com/paper/2405.00838/) (2024)

## Abstract (arXiv)

Multimodal ophthalmic imaging-based diagnosis integrates color fundus image with optical coherence tomography (OCT) to provide a comprehensive view of ocular pathologies. However, the uneven global distribution of healthcare resources often results in real-world clinical scenarios encountering incomplete multimodal data, which significantly compromises diagnostic accuracy. Existing commonly used pipelines, such as modality imputation and distillation methods, face notable limitations: 1)Imputation methods struggle with accurately reconstructing key lesion features, since OCT lesions are localized, while fundus images vary in style. 2)distillation methods rely heavily on fully paired multimodal training data. To address these challenges, we propose a novel multimodal alignment and fusion framework capable of robustly handling missing modalities in the task of ophthalmic diagnostics. By considering the distinctive feature characteristics of OCT and fundus images, we emphasize the alignment of semantic features within the same category and explicitly learn soft matching between modalities, allowing the missing modality to utilize existing modality information, achieving robust cross-modal feature alignment under the missing modality. Specifically, we leverage the Optimal Transport for multi-scale modality feature alignment: class-wise alignment through predicted class prototypes and feature-wise alignment via cross-modal shared feature transport. Furthermore, we propose an asymmetric fusion strategy that effectively exploits the distinct characteristics of OCT and fundus modalities. Extensive evaluations on three large ophthalmic multimodal datasets demonstrate our model's superior performance under various modality-incomplete scenarios, achieving Sota performance in both complete modality and inter-modality incompleteness conditions. Code is available at https://github.com/Qinkaiyu/RIMA