# Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)

arXiv: https://arxiv.org/abs/2603.17715 · HTML: https://mlsanity.com/paper/2603.17715/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 59 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles](https://mlsanity.com/paper/2306.00989/) (2023)
- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities](https://mlsanity.com/paper/1905.13146/) (2019)
- [Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation](https://mlsanity.com/paper/2601.10880/) (2026)
- [EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking](https://mlsanity.com/paper/2007.09600/) (2020)
- [PupilNet: Convolutional Neural Networks for Robust Pupil Detection](https://mlsanity.com/paper/1601.04902/) (2016)
- [Window Attention is Bugged: How not to Interpolate Position Embeddings](https://mlsanity.com/paper/2311.05613/) (2023)
- [PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection](https://mlsanity.com/paper/1711.00112/) (2017)
- [Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gaze](https://mlsanity.com/paper/2201.06799/) (2022)
- [Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)](https://mlsanity.com/paper/2311.08077/) (2023)
- [LEyes: A Lightweight Framework for Deep Learning-Based Eye Tracking using Synthetic Eye Images](https://mlsanity.com/paper/2309.06129/) (2023)
- [Towards Unsupervised Eye-Region Segmentation for Eye Tracking](https://mlsanity.com/paper/2410.06131/) (2024)
- [Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images](https://mlsanity.com/paper/2410.08926/) (2024)
- [Comparing SAM 2 and SAM 3 for Zero-Shot Segmentation of 3D Medical Data](https://mlsanity.com/paper/2511.21926/) (2025)

## Abstract (arXiv)

Previous work has reported that vision foundation models show promising zero-shot performance in eye image segmentation. Here we examine whether the latest iteration of the Segment Anything Model, SAM3, offers better eye image segmentation performance than SAM2, and explore the performance of its new concept (text) prompting mode. Eye image segmentation performance was evaluated using diverse datasets encompassing both high-resolution high-quality videos from a lab environment and the TEyeD dataset consisting of challenging eye videos acquired in the wild. Results show that in most cases SAM3 with either visual or concept prompts did not perform better than SAM2, for both lab and in-the-wild datasets. Since SAM2 not only performed better but was also faster, we conclude that SAM2 remains the best option for eye image segmentation. We provide our adaptation of SAM3's codebase that allows processing videos of arbitrary duration.