# TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types

arXiv: https://arxiv.org/abs/2102.02115 · HTML: https://mlsanity.com/paper/2102.02115/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 99 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities](https://mlsanity.com/paper/1905.13146/) (2019)
- [PupilNet: Convolutional Neural Networks for Robust Pupil Detection](https://mlsanity.com/paper/1601.04902/) (2016)
- [Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments](https://mlsanity.com/paper/1511.05768/) (2015)
- [ElSe: Ellipse Selection for Robust Pupil Detection in Real-World Environments](https://mlsanity.com/paper/1511.06575/) (2015)
- [PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection](https://mlsanity.com/paper/1711.00112/) (2017)
- [Fully Convolutional Neural Networks for Raw Eye Tracking Data Segmentation, Generation, and Reconstruction](https://mlsanity.com/paper/2002.10905/) (2020)
- [Rotated Ring, Radial and Depth Wise Separable Radial Convolutions](https://mlsanity.com/paper/2010.00873/) (2020)
- [Eye movement velocity and gaze data generator for evaluation, robustness testing and assess of eye tracking software and visualization tools](https://mlsanity.com/paper/1808.09296/) (2018)
- [Learning to Validate the Quality of Detected Landmarks](https://mlsanity.com/paper/1901.10143/) (2019)
- [Weight and Gradient Centralization in Deep Neural Networks](https://mlsanity.com/paper/2010.00866/) (2020)
- [Fast camera focus estimation for gaze-based focus control](https://mlsanity.com/paper/1711.03306/) (2017)
- [Eye movement simulation and detector creation to reduce laborious parameter adjustments](https://mlsanity.com/paper/1804.00970/) (2018)
- [Training Decision Trees as Replacement for Convolution Layers](https://mlsanity.com/paper/1905.10073/) (2019)
- [The Gaze and Mouse Signal as additional Source for User Fingerprints in Browser Applications](https://mlsanity.com/paper/2101.03793/) (2021)
- [Multi Layer Neural Networks as Replacement for Pooling Operations](https://mlsanity.com/paper/2006.06969/) (2020)
- [Explainable Online Validation of Machine Learning Models for Practical Applications](https://mlsanity.com/paper/2010.00821/) (2020)
- [A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis](https://mlsanity.com/paper/2101.04318/) (2021)
- [Bayesian Identification of Fixations, Saccades, and Smooth Pursuits](https://mlsanity.com/paper/1511.07732/) (2015)
- [MagicEyes: A Large Scale Eye Gaze Estimation Dataset for Mixed Reality](https://mlsanity.com/paper/2003.08806/) (2020)

## Built on by (most cited first, 30 of 32)

- [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023)
- [Event-Based Eye Tracking. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.11770/) (2024)
- [EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes](https://mlsanity.com/paper/2206.08428/) (2022)
- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges](https://mlsanity.com/paper/2305.14080/) (2023)
- [3DGazeNet: Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views](https://mlsanity.com/paper/2212.02997/) (2022)
- [Pistol: Pupil Invisible Supportive Tool to extract Pupil, Iris, Eye Opening, Eye Movements, Pupil and Iris Gaze Vector, and 2D as well as 3D Gaze](https://mlsanity.com/paper/2201.06799/) (2022)
- [Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN](https://mlsanity.com/paper/2404.14279/) (2024)
- [Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)](https://mlsanity.com/paper/2311.08077/) (2023)
- [Maximum and Leaky Maximum Propagation](https://mlsanity.com/paper/2105.10277/) (2021)
- [LEyes: A Lightweight Framework for Deep Learning-Based Eye Tracking using Synthetic Eye Images](https://mlsanity.com/paper/2309.06129/) (2023)
- [The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video](https://mlsanity.com/paper/2404.18934/) (2024)
- [Tensor Normalization and Full Distribution Training](https://mlsanity.com/paper/2109.02345/) (2021)
- [GroupGazer: A Tool to Compute the Gaze per Participant in Groups with integrated Calibration to Map the Gaze Online to a Screen or Beamer Projection](https://mlsanity.com/paper/2201.07692/) (2022)
- [PupilSense: A Novel Application for Webcam-Based Pupil Diameter Estimation](https://mlsanity.com/paper/2407.11204/) (2024)
- [CondSeg: Ellipse Estimation of Pupil and Iris via Conditioned Segmentation](https://mlsanity.com/paper/2408.17231/) (2024)
- [Towards Unsupervised Eye-Region Segmentation for Eye Tracking](https://mlsanity.com/paper/2410.06131/) (2024)
- [Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images](https://mlsanity.com/paper/2410.08926/) (2024)
- [Artificial Neural Networks for Finger Vein Recognition: A Survey](https://mlsanity.com/paper/2208.13341/) (2022)
- [Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions](https://mlsanity.com/paper/2311.12157/) (2023)
- [GazeShift: Unsupervised Gaze Estimation and Dataset for VR](https://mlsanity.com/paper/2603.07832/) (2026)
- [Rapidly deploying on-device eye tracking by distilling visual foundation models](https://mlsanity.com/paper/2604.02509/) (2026)
- [On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Array](https://mlsanity.com/paper/2202.00836/) (2022)
- [Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets](https://mlsanity.com/paper/2206.09697/) (2022)
- [DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery](https://mlsanity.com/paper/2311.07235/) (2023)
- [Normalized Validity Scores for DNNs in Regression based Eye Feature Extraction](https://mlsanity.com/paper/2403.11665/) (2024)
- [EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR](https://mlsanity.com/paper/2507.09649/) (2025)
- [Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)](https://mlsanity.com/paper/2603.17715/) (2026)
- [PicoEyes: Unified Gaze Estimation Framework for Mixed Reality with a Large-Scale Multi-View Dataset](https://mlsanity.com/paper/2605.07188/) (2026)
- [AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination](https://mlsanity.com/paper/2606.03774/) (2026)

## Abstract (arXiv)

We present TEyeD, the world's largest unified public data set of eye images taken with head-mounted devices. TEyeD was acquired with seven different head-mounted eye trackers. Among them, two eye trackers were integrated into virtual reality (VR) or augmented reality (AR) devices. The images in TEyeD were obtained from various tasks, including car rides, simulator rides, outdoor sports activities, and daily indoor activities. The data set includes 2D and 3D landmarks, semantic segmentation, 3D eyeball annotation and the gaze vector and eye movement types for all images. Landmarks and semantic segmentation are provided for the pupil, iris and eyelids. Video lengths vary from a few minutes to several hours. With more than 20 million carefully annotated images, TEyeD provides a unique, coherent resource and a valuable foundation for advancing research in the field of computer vision, eye tracking and gaze estimation in modern VR and AR applications. Download: https://es-cloud.cs.uni-tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FTEyeDS&mode=list Alternative Download: https://hctlsrva.edu.sot.tum.de/TEyeDS/