# Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality

arXiv: https://arxiv.org/abs/2403.19768 · HTML: https://mlsanity.com/paper/2403.19768/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 28 Mar 2024.

## Builds on (most cited first, 11 of 11)

- [Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction](https://mlsanity.com/paper/1405.0006/) (2014)
- [RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking](https://mlsanity.com/paper/1910.00694/) (2019)
- [EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking](https://mlsanity.com/paper/2007.09600/) (2020)
- [PupilNet: Convolutional Neural Networks for Robust Pupil Detection](https://mlsanity.com/paper/1601.04902/) (2016)
- [A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms](https://mlsanity.com/paper/1708.01817/) (2017)
- [Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments](https://mlsanity.com/paper/1511.05768/) (2015)
- [PuRe: Robust pupil detection for real-time pervasive eye tracking](https://mlsanity.com/paper/1712.08900/) (2017)
- [ElSe: Ellipse Selection for Robust Pupil Detection in Real-World Environments](https://mlsanity.com/paper/1511.06575/) (2015)
- [RIT-Eyes: Rendering of near-eye images for eye-tracking applications](https://mlsanity.com/paper/2006.03642/) (2020)
- [1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning](https://mlsanity.com/paper/2102.01921/) (2021)
- [EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking](https://mlsanity.com/paper/2205.01947/) (2022)

## Built on by (most cited first, 1 of 1)

- [Low Latency Gaze Tracking via Latent Optical Sensing](https://mlsanity.com/paper/2605.17990/) (2026)

## Abstract (arXiv)

Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.