# EFE: End-to-end Frame-to-Gaze Estimation

arXiv: https://arxiv.org/abs/2305.05526 · HTML: https://mlsanity.com/paper/2305.05526/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 20 of 20)

- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1611.08860/) (2016)
- [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017)
- [Few-Shot Adaptive Gaze Estimation](https://mlsanity.com/paper/1905.01941/) (2019)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Deep Pictorial Gaze Estimation](https://mlsanity.com/paper/1807.10002/) (2018)
- [Towards End-to-end Video-based Eye-Tracking](https://mlsanity.com/paper/2007.13120/) (2020)
- [Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications](https://mlsanity.com/paper/1901.10906/) (2019)
- [Adaptive Feature Fusion Network for Gaze Tracking in Mobile Tablets](https://mlsanity.com/paper/2103.11119/) (2021)
- [Weakly-Supervised Physically Unconstrained Gaze Estimation](https://mlsanity.com/paper/2105.09803/) (2021)
- [GazeOnce: Real-Time Multi-Person Gaze Estimation](https://mlsanity.com/paper/2204.09480/) (2022)
- [A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone](https://mlsanity.com/paper/1910.07331/) (2019)
- [Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation](https://mlsanity.com/paper/2007.13090/) (2020)

## Built on by (most cited first, 11 of 11)

- [Quantifying the Impact of Motion on 2D Gaze Estimation in Real-World Mobile Interactions](https://mlsanity.com/paper/2502.10570/) (2025)
- [3D Prior is All You Need: Cross-Task Few-shot 2D Gaze Estimation](https://mlsanity.com/paper/2502.04074/) (2025)
- [Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions](https://mlsanity.com/paper/2311.12157/) (2023)
- [EraW-Net: Enhance-Refine-Align W-Net for Scene-Associated Driver Attention Estimation](https://mlsanity.com/paper/2408.08570/) (2024)
- [WEBEYETRACK: Scalable Eye-Tracking for the Browser via On-Device Few-Shot Personalization](https://mlsanity.com/paper/2508.19544/) (2025)
- [Investigating Bias and Fairness in Appearance-based Gaze Estimation](https://mlsanity.com/paper/2604.10707/) (2026)
- [See Where You Read with Eye Gaze Tracking and Large Language Model](https://mlsanity.com/paper/2409.19454/) (2024)
- [Lightweight Gaze Estimation Model Via Fusion Global Information](https://mlsanity.com/paper/2411.18064/) (2024)
- [Pose-Robust Calibration Strategy for Point-of-Gaze Estimation on Mobile Phones](https://mlsanity.com/paper/2508.10268/) (2025)
- [Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction](https://mlsanity.com/paper/2605.04770/) (2026)
- [Drift Calibration in Geometric Eye Tracking Systems](https://mlsanity.com/paper/2608.29739/) (2026)

## Abstract (arXiv)

Despite the recent development of learning-based gaze estimation methods, most methods require one or more eye or face region crops as inputs and produce a gaze direction vector as output. Cropping results in a higher resolution in the eye regions and having fewer confounding factors (such as clothing and hair) is believed to benefit the final model performance. However, this eye/face patch cropping process is expensive, erroneous, and implementation-specific for different methods. In this paper, we propose a frame-to-gaze network that directly predicts both 3D gaze origin and 3D gaze direction from the raw frame out of the camera without any face or eye cropping. Our method demonstrates that direct gaze regression from the raw downscaled frame, from FHD/HD to VGA/HVGA resolution, is possible despite the challenges of having very few pixels in the eye region. The proposed method achieves comparable results to state-of-the-art methods in Point-of-Gaze (PoG) estimation on three public gaze datasets: GazeCapture, MPIIFaceGaze, and EVE, and generalizes well to extreme camera view changes.