# Towards End-to-end Video-based Eye-Tracking

arXiv: https://arxiv.org/abs/2007.13120 · HTML: https://mlsanity.com/paper/2007.13120/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2020.

## Built on by (most cited first, 30 of 39)

- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [Weakly-Supervised Physically Unconstrained Gaze Estimation](https://mlsanity.com/paper/2105.09803/) (2021)
- [DVGaze: Dual-View Gaze Estimation](https://mlsanity.com/paper/2308.10310/) (2023)
- [Contrastive Representation Learning for Gaze Estimation](https://mlsanity.com/paper/2210.13404/) (2022)
- [What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation](https://mlsanity.com/paper/2403.15664/) (2024)
- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Spatio-Temporal Attention and Gaussian Processes for Personalized Video Gaze Estimation](https://mlsanity.com/paper/2404.05215/) (2024)
- [3DGazeNet: Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views](https://mlsanity.com/paper/2212.02997/) (2022)
- [EFE: End-to-end Frame-to-Gaze Estimation](https://mlsanity.com/paper/2305.05526/) (2023)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [Generalizable Deepfake Detection with Phase-Based Motion Analysis](https://mlsanity.com/paper/2211.09363/) (2022)
- [Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels](https://mlsanity.com/paper/2502.20249/) (2025)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation](https://mlsanity.com/paper/2106.14183/) (2021)
- [One Eye is All You Need: Lightweight Ensembles for Gaze Estimation with Single Encoders](https://mlsanity.com/paper/2211.11936/) (2022)
- [Multi-view Gaze Target Estimation](https://mlsanity.com/paper/2508.05857/) (2025)
- [Semi-Supervised Learning for Eye Image Segmentation](https://mlsanity.com/paper/2103.09369/) (2021)
- [Modern Augmented Reality: Applications, Trends, and Future Directions](https://mlsanity.com/paper/2202.09450/) (2022)
- [A Novel Framework for Multi-Person Temporal Gaze Following and Social Gaze Prediction](https://mlsanity.com/paper/2403.10511/) (2024)
- [Evaluating Image-Based Face and Eye Tracking with Event Cameras](https://mlsanity.com/paper/2408.10395/) (2024)
- [3D Prior is All You Need: Cross-Task Few-shot 2D Gaze Estimation](https://mlsanity.com/paper/2502.04074/) (2025)
- [Multistream Gaze Estimation with Anatomical Eye Region Isolation by Synthetic to Real Transfer Learning](https://mlsanity.com/paper/2206.09256/) (2022)
- [Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions](https://mlsanity.com/paper/2311.12157/) (2023)
- [Driver Attention Tracking and Analysis](https://mlsanity.com/paper/2404.07122/) (2024)
- [Smartphone-based Eye Tracking System using Edge Intelligence and Model Optimisation](https://mlsanity.com/paper/2408.12463/) (2024)
- [A Survey on Deep Learning-based Gaze Direction Regression: Searching for the State-of-the-art](https://mlsanity.com/paper/2410.17082/) (2024)
- [Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform](https://mlsanity.com/paper/2509.24001/) (2025)
- [Detection of Abnormal Behavior with Self-Supervised Gaze Estimation](https://mlsanity.com/paper/2107.06530/) (2021)
- [RAZE: Region Guided Self-Supervised Gaze Representation Learning](https://mlsanity.com/paper/2208.02485/) (2022)

## Abstract (arXiv)

Estimating eye-gaze from images alone is a challenging task, in large parts due to un-observable person-specific factors. Achieving high accuracy typically requires labeled data from test users which may not be attainable in real applications. We observe that there exists a strong relationship between what users are looking at and the appearance of the user's eyes. In response to this understanding, we propose a novel dataset and accompanying method which aims to explicitly learn these semantic and temporal relationships. Our video dataset consists of time-synchronized screen recordings, user-facing camera views, and eye gaze data, which allows for new benchmarks in temporal gaze tracking as well as label-free refinement of gaze. Importantly, we demonstrate that the fusion of information from visual stimuli as well as eye images can lead towards achieving performance similar to literature-reported figures acquired through supervised personalization. Our final method yields significant performance improvements on our proposed EVE dataset, with up to a 28 percent improvement in Point-of-Gaze estimates (resulting in 2.49 degrees in angular error), paving the path towards high-accuracy screen-based eye tracking purely from webcam sensors. The dataset and reference source code are available at https://ait.ethz.ch/projects/2020/EVE