# Cross-Fundus Transformer for Multi-modal Diabetic Retinopathy Grading with Cataract

arXiv: https://arxiv.org/abs/2411.00726 · HTML: https://mlsanity.com/paper/2411.00726/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks](https://mlsanity.com/paper/1804.07839/) (2018)
- [Quantifying the Value of Lateral Views in Deep Learning for Chest X-rays](https://mlsanity.com/paper/2002.02582/) (2020)
- [Multi-Modal Multi-Instance Learning for Retinal Disease Recognition](https://mlsanity.com/paper/2109.12307/) (2021)
- [Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization](https://mlsanity.com/paper/1907.12023/) (2019)
- [Cross-Field Transformer for Diabetic Retinopathy Grading on Two-field Fundus Images](https://mlsanity.com/paper/2211.14552/) (2022)

## Built on by (most cited first, 1 of 1)

- [Multi-modal and Multi-view Fundus Image Fusion for Retinopathy Diagnosis via Multi-scale Cross-attention and Shifted Window Self-attention](https://mlsanity.com/paper/2504.09106/) (2025)

## Abstract (arXiv)

Diabetic retinopathy (DR) is a leading cause of blindness worldwide and a common complication of diabetes. As two different imaging tools for DR grading, color fundus photography (CFP) and infrared fundus photography (IFP) are highly-correlated and complementary in clinical applications. To the best of our knowledge, this is the first study that explores a novel multi-modal deep learning framework to fuse the information from CFP and IFP towards more accurate DR grading. Specifically, we construct a dual-stream architecture Cross-Fundus Transformer (CFT) to fuse the ViT-based features of two fundus image modalities. In particular, a meticulously engineered Cross-Fundus Attention (CFA) module is introduced to capture the correspondence between CFP and IFP images. Moreover, we adopt both the single-modality and multi-modality supervisions to maximize the overall performance for DR grading. Extensive experiments on a clinical dataset consisting of 1,713 pairs of multi-modal fundus images demonstrate the superiority of our proposed method. Our code will be released for public access.