# GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy

arXiv: https://arxiv.org/abs/2311.08075 · HTML: https://mlsanity.com/paper/2311.08075/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 17 of 17)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [Large AI Models in Health Informatics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2303.11568/) (2023)
- [Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development](https://mlsanity.com/paper/2009.07386/) (2020)
- [Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation](https://mlsanity.com/paper/2304.12637/) (2023)
- [Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis](https://mlsanity.com/paper/2204.02976/) (2022)
- [Learn to Segment Retinal Lesions and Beyond](https://mlsanity.com/paper/1912.11619/) (2019)
- [Deep image mining for diabetic retinopathy screening](https://mlsanity.com/paper/1610.07086/) (2016)
- [VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence](https://mlsanity.com/paper/2310.04992/) (2023)
- [Learnable Ophthalmology SAM](https://mlsanity.com/paper/2304.13425/) (2023)
- [GazeSAM: What You See is What You Segment](https://mlsanity.com/paper/2304.13844/) (2023)
- [RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation](https://mlsanity.com/paper/2201.11037/) (2022)
- [mathrmSAMMed: A medical image annotation framework based on large vision model](https://mlsanity.com/paper/2307.05617/) (2023)
- [CauDR: A Causality-inspired Domain Generalization Framework for Fundus-based Diabetic Retinopathy Grading](https://mlsanity.com/paper/2309.15493/) (2023)

## Built on by (most cited first, 4 of 4)

- [From Gaze to Insight: Bridging Human Visual Attention and Vision Language Model Explanation for Weakly-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2504.11368/) (2025)
- [Minuscule Cell Detection in AS-OCT Images with Progressive Field-of-View Focusing](https://mlsanity.com/paper/2503.12249/) (2025)
- [Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation](https://mlsanity.com/paper/2505.22230/) (2025)
- [Rapid Bone Scintigraphy Enhancement via Semantic Prior Distillation from Segment Anything Model](https://mlsanity.com/paper/2503.02321/) (2025)

## Abstract (arXiv)

Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microangioma lesions, resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of microangioma lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze map, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting microangioma lesions. Finally, a domain knowledge filter refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and enhances segmentation performance through fine-tuning using annotations. This study highlights the potential of GlanceSeg-based annotations for self-model optimization, leading to enduring performance advancements through continual learning.