# Automatic Gaze Analysis: A Survey of Deep Learning based Approaches

arXiv: https://arxiv.org/abs/2108.05479 · HTML: https://mlsanity.com/paper/2108.05479/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 243 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 30 of 69)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1611.08860/) (2016)
- [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017)
- [DSFD: Dual Shot Face Detector](https://mlsanity.com/paper/1810.10220/) (2018)
- [Few-Shot Adaptive Gaze Estimation](https://mlsanity.com/paper/1905.01941/) (2019)
- [Detecting Attended Visual Targets in Video](https://mlsanity.com/paper/2003.02501/) (2020)
- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation](https://mlsanity.com/paper/2001.00187/) (2020)
- [PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation](https://mlsanity.com/paper/2103.13173/) (2021)
- [Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency](https://mlsanity.com/paper/1807.10437/) (2018)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Deep Pictorial Gaze Estimation](https://mlsanity.com/paper/1807.10002/) (2018)
- [FaceX-Zoo: A PyTorch Toolbox for Face Recognition](https://mlsanity.com/paper/2101.04407/) (2021)
- [Appearance-Based Gaze Estimation Using Dilated-Convolutions](https://mlsanity.com/paper/1903.07296/) (2019)
- [Improving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis](https://mlsanity.com/paper/1904.10638/) (2019)
- [Unsupervised Representation Learning for Gaze Estimation](https://mlsanity.com/paper/1911.06939/) (2019)
- [Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning](https://mlsanity.com/paper/1909.02144/) (2019)
- [Towards End-to-end Video-based Eye-Tracking](https://mlsanity.com/paper/2007.13120/) (2020)
- [Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings](https://mlsanity.com/paper/1805.04771/) (2018)
- [LAEO-Net: revisiting people Looking At Each Other in videos](https://mlsanity.com/paper/1906.05261/) (2019)
- [Prediction and Localization of Student Engagement in the Wild](https://mlsanity.com/paper/1804.00858/) (2018)

## Built on by (most cited first, 30 of 33)

- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [Contrastive Representation Learning for Gaze Estimation](https://mlsanity.com/paper/2210.13404/) (2022)
- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [Rotation-Constrained Cross-View Feature Fusion for Multi-View Appearance-based Gaze Estimation](https://mlsanity.com/paper/2305.12704/) (2023)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [GazeVLM: A Vision-Language Model for Multi-Task Gaze Understanding](https://mlsanity.com/paper/2511.06348/) (2025)
- [UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training](https://mlsanity.com/paper/2502.02307/) (2025)
- [CrossGaze: A Strong Method for 3D Gaze Estimation in the Wild](https://mlsanity.com/paper/2402.08316/) (2024)
- [Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems](https://mlsanity.com/paper/2403.15947/) (2024)
- [Quantifying the Impact of Motion on 2D Gaze Estimation in Real-World Mobile Interactions](https://mlsanity.com/paper/2502.10570/) (2025)
- [From Gaze to Insight: Bridging Human Visual Attention and Vision Language Model Explanation for Weakly-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2504.11368/) (2025)
- [Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach](https://mlsanity.com/paper/2311.05567/) (2023)
- [Seeing Through Deepfakes: A Human-Inspired Framework for Multi-Face Detection](https://mlsanity.com/paper/2507.14807/) (2025)
- [AV-Gaze: A Study on the Effectiveness of Audio Guided Visual Attention Estimation for Non-Profilic Faces](https://mlsanity.com/paper/2207.03048/) (2022)
- [Investigation of Architectures and Receptive Fields for Appearance-based Gaze Estimation](https://mlsanity.com/paper/2308.09593/) (2023)
- [Digitally Prototype Your Eye Tracker: Simulating Hardware Performance using 3D Synthetic Data](https://mlsanity.com/paper/2503.16742/) (2025)
- [Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation](https://mlsanity.com/paper/2505.22230/) (2025)
- [Unsupervised Gaze-aware Contrastive Learning with Subject-specific Condition](https://mlsanity.com/paper/2309.04506/) (2023)
- [Upper-Body Pose-based Gaze Estimation for Privacy-Preserving 3D Gaze Target Detection](https://mlsanity.com/paper/2409.17886/) (2024)
- [A Survey on Deep Learning-based Gaze Direction Regression: Searching for the State-of-the-art](https://mlsanity.com/paper/2410.17082/) (2024)
- [Investigating Bias and Fairness in Appearance-based Gaze Estimation](https://mlsanity.com/paper/2604.10707/) (2026)
- [OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision](https://mlsanity.com/paper/2606.07431/) (2026)
- ['Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation](https://mlsanity.com/paper/2208.01840/) (2022)
- [RAZE: Region Guided Self-Supervised Gaze Representation Learning](https://mlsanity.com/paper/2208.02485/) (2022)
- [Contrastive Weighted Learning for Near-Infrared Gaze Estimation](https://mlsanity.com/paper/2211.03073/) (2022)
- [Angle Range and Identity Similarity Enhanced Gaze and Head Redirection based on Synthetic data](https://mlsanity.com/paper/2309.05214/) (2023)
- [Stable Tracking of Eye Gaze Direction During Ophthalmic Surgery](https://mlsanity.com/paper/2507.00635/) (2025)
- [PrivateXR: Defending Privacy Attacks in Extended Reality Through Explainable AI-Guided Differential Privacy](https://mlsanity.com/paper/2512.16851/) (2025)

## Abstract (arXiv)

Eye gaze analysis is an important research problem in the field of Computer Vision and Human-Computer Interaction. Even with notable progress in the last 10 years, automatic gaze analysis still remains challenging due to the uniqueness of eye appearance, eye-head interplay, occlusion, image quality, and illumination conditions. There are several open questions, including what are the important cues to interpret gaze direction in an unconstrained environment without prior knowledge and how to encode them in real-time. We review the progress across a range of gaze analysis tasks and applications to elucidate these fundamental questions, identify effective methods in gaze analysis, and provide possible future directions. We analyze recent gaze estimation and segmentation methods, especially in the unsupervised and weakly supervised domain, based on their advantages and reported evaluation metrics. Our analysis shows that the development of a robust and generic gaze analysis method still needs to address real-world challenges such as unconstrained setup and learning with less supervision. We conclude by discussing future research directions for designing a real-world gaze analysis system that can propagate to other domains including Computer Vision, Augmented Reality (AR), Virtual Reality (VR), and Human Computer Interaction (HCI). Project Page: https://github.com/i-am-shreya/EyeGazeSurvey}{https://github.com/i-am-shreya/EyeGazeSurvey