# GazeSAM: What You See is What You Segment

arXiv: https://arxiv.org/abs/2304.13844 · HTML: https://mlsanity.com/paper/2304.13844/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 10 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 2 of 2)

- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [A Collaborative Computer Aided Diagnosis (C-CAD) System with Eye-Tracking, Sparse Attentional Model, and Deep Learning](https://mlsanity.com/paper/1802.06260/) (2018)

## Built on by (most cited first, 19 of 19)

- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023)
- [GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classification](https://mlsanity.com/paper/2305.18221/) (2023)
- [Towards Segment Anything Model (SAM) for Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.03678/) (2023)
- [A Survey on Trustworthiness in Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2407.15851/) (2024)
- [Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities](https://mlsanity.com/paper/2406.08379/) (2024)
- [Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)](https://mlsanity.com/paper/2311.08077/) (2023)
- [Integrating Travel Behavior Forecasting and Generative Modeling for Predicting Future Urban Mobility and Spatial Transformations](https://mlsanity.com/paper/2503.21158/) (2025)
- [GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy](https://mlsanity.com/paper/2311.08075/) (2023)
- [RadGazeGen: Radiomics and Gaze-guided Medical Image Generation using Diffusion Models](https://mlsanity.com/paper/2410.00307/) (2024)
- [Gaze-DETR: Using Expert Gaze to Reduce False Positives in Vulvovaginal Candidiasis Screening](https://mlsanity.com/paper/2405.09463/) (2024)
- [Zero-Shot Gaze-based Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2505.15256/) (2025)
- [Vision-language models for decoding provider attention during neonatal resuscitation](https://mlsanity.com/paper/2404.01207/) (2024)
- [GazeLT: Visual attention-guided long-tailed disease classification in chest radiographs](https://mlsanity.com/paper/2508.09478/) (2025)
- [Eye on the Target: Eye Tracking Meets Rodent Tracking](https://mlsanity.com/paper/2503.10305/) (2025)
- [GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation](https://mlsanity.com/paper/2609.01310/) (2026)

## Abstract (arXiv)

This study investigates the potential of eye-tracking technology and the Segment Anything Model (SAM) to design a collaborative human-computer interaction system that automates medical image segmentation. We present the GazeSAM system to enable radiologists to collect segmentation masks by simply looking at the region of interest during image diagnosis. The proposed system tracks radiologists' eye movement and utilizes the eye-gaze data as the input prompt for SAM, which automatically generates the segmentation mask in real time. This study is the first work to leverage the power of eye-tracking technology and SAM to enhance the efficiency of daily clinical practice. Moreover, eye-gaze data coupled with image and corresponding segmentation labels can be easily recorded for further advanced eye-tracking research. The code is available in urlhttps://github.com/ukaukaaaa/GazeSAM.