# Towards Unsupervised Eye-Region Segmentation for Eye Tracking

arXiv: https://arxiv.org/abs/2410.06131 · HTML: https://mlsanity.com/paper/2410.06131/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [Semi-supervised Semantic Segmentation with Directional Context-aware Consistency](https://mlsanity.com/paper/2106.14133/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis](https://mlsanity.com/paper/2204.02976/) (2022)
- [RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking](https://mlsanity.com/paper/1910.00694/) (2019)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [OpenEDS: Open Eye Dataset](https://mlsanity.com/paper/1905.03702/) (2019)
- [Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities](https://mlsanity.com/paper/1905.13146/) (2019)
- [EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking](https://mlsanity.com/paper/2007.09600/) (2020)
- [OpenEDS2020: Open Eyes Dataset](https://mlsanity.com/paper/2005.03876/) (2020)
- [PuRe: Robust pupil detection for real-time pervasive eye tracking](https://mlsanity.com/paper/1712.08900/) (2017)
- [1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning](https://mlsanity.com/paper/2102.01921/) (2021)
- [Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)](https://mlsanity.com/paper/2311.08077/) (2023)
- [Eye Semantic Segmentation with a Lightweight Model](https://mlsanity.com/paper/1911.01049/) (2019)
- [Semi-Supervised Learning for Eye Image Segmentation](https://mlsanity.com/paper/2103.09369/) (2021)

## Built on by (most cited first, 2 of 2)

- [Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images](https://mlsanity.com/paper/2410.08926/) (2024)
- [Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)](https://mlsanity.com/paper/2603.17715/) (2026)

## Abstract (arXiv)

Finding the eye and parsing out the parts (e.g. pupil and iris) is a key prerequisite for image-based eye tracking, which has become an indispensable module in today's head-mounted VR/AR devices. However, a typical route for training a segmenter requires tedious handlabeling. In this work, we explore an unsupervised way. First, we utilize priors of human eye and extract signals from the image to establish rough clues indicating the eye-region structure. Upon these sparse and noisy clues, a segmentation network is trained to gradually identify the precise area for each part. To achieve accurate parsing of the eye-region, we first leverage the pretrained foundation model Segment Anything (SAM) in an automatic way to refine the eye indications. Then, the learning process is designed in an end-to-end manner following progressive and prior-aware principle. Experiments show that our unsupervised approach can easily achieve 90% (the pupil and iris) and 85% (the whole eye-region) of the performances under supervised learning.