# CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Prompting

arXiv: https://arxiv.org/abs/2407.04068 · HTML: https://mlsanity.com/paper/2407.04068/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2024.

## Built on by (most cited first, 18 of 18)

- [Pathology-Aware Prototype Evolution via LLM-Driven Semantic Disambiguation for Multicenter Diabetic Retinopathy Diagnosis](https://mlsanity.com/paper/2511.22033/) (2025)
- [Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2506.18378/) (2025)
- [A Survey on Ordinal Regression: Applications, Advances and Prospects](https://mlsanity.com/paper/2503.00952/) (2025)
- [Parameterized Diffusion Optimization enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading](https://mlsanity.com/paper/2507.04978/) (2025)
- [Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification](https://mlsanity.com/paper/2511.16845/) (2025)
- [Ranking-aware Reinforcement Learning for Ordinal Ranking](https://mlsanity.com/paper/2601.20585/) (2026)
- [DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression](https://mlsanity.com/paper/2606.07599/) (2026)
- [Facial Attractiveness Prediction in Live Streaming: A New Benchmark and Multi-modal Method](https://mlsanity.com/paper/2501.02509/) (2025)
- [CLIP-KOA: Enhancing Knee Osteoarthritis Diagnosis with Multi-Modal Learning and Symmetry-Aware Loss Functions](https://mlsanity.com/paper/2504.19443/) (2025)
- [TWLR: Text-Guided Weakly-Supervised Lesion Localization and Severity Regression for Explainable Diabetic Retinopathy Grading](https://mlsanity.com/paper/2512.13008/) (2025)
- [Does medical specialization of VLMs enhance discriminative power?: A comprehensive investigation through feature distribution analysis](https://mlsanity.com/paper/2601.14774/) (2026)
- [Experience with Single Domain Generalization in Real World Medical Imaging Deployments](https://mlsanity.com/paper/2601.16359/) (2026)
- [Directed Ordinal Diffusion Regularization for Progression-Aware Diabetic Retinopathy Grading](https://mlsanity.com/paper/2602.21942/) (2026)
- [Ordinal Diffusion Models for Color Fundus Images](https://mlsanity.com/paper/2602.24013/) (2026)
- [Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization](https://mlsanity.com/paper/2603.12369/) (2026)
- [Diabetic Retinopathy Grading with CLIP-based Ranking-Aware Adaptation:A Comparative Study on Fundus Image](https://mlsanity.com/paper/2603.13403/) (2026)
- [Scientific Domain Knowledge Improves Vision-Language Fundus Models](https://mlsanity.com/paper/2605.02720/) (2026)
- [Cross-Modal Semantic-Enhanced Diffusion Framework for Diabetic Retinopathy Grading](https://mlsanity.com/paper/2605.09242/) (2026)

## Abstract (arXiv)

Diabetic retinopathy (DR) is a complication of diabetes and usually takes decades to reach sight-threatening levels. Accurate and robust detection of DR severity is critical for the timely management and treatment of diabetes. However, most current DR grading methods suffer from insufficient robustness to data variability (textite.g. colour fundus images), posing a significant difficulty for accurate and robust grading. In this work, we propose a novel DR grading framework CLIP-DR based on three observations: 1) Recent pre-trained visual language models, such as CLIP, showcase a notable capacity for generalisation across various downstream tasks, serving as effective baseline models. 2) The grading of image-text pairs for DR often adheres to a discernible natural sequence, yet most existing DR grading methods have primarily overlooked this aspect. 3) A long-tailed distribution among DR severity levels complicates the grading process. This work proposes a novel ranking-aware prompting strategy to help the CLIP model exploit the ordinal information. Specifically, we sequentially design learnable prompts between neighbouring text-image pairs in two different ranking directions. Additionally, we introduce a Similarity Matrix Smooth module into the structure of CLIP to balance the class distribution. Finally, we perform extensive comparisons with several state-of-the-art methods on the GDRBench benchmark, demonstrating our CLIP-DR's robustness and superior performance. The implementation code is available footnoteurlhttps://github.com/Qinkaiyu/CLIP-DR