# 3DGazeNet: Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views

arXiv: https://arxiv.org/abs/2212.02997 · HTML: https://mlsanity.com/paper/2212.02997/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 86 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild](https://mlsanity.com/paper/2004.01946/) (2020)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars](https://mlsanity.com/paper/2211.11208/) (2022)
- [It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1611.08860/) (2016)
- [Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars](https://mlsanity.com/paper/2008.10174/) (2020)
- [HeadGAN: One-shot Neural Head Synthesis and Editing](https://mlsanity.com/paper/2012.08261/) (2020)
- [GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds](https://mlsanity.com/paper/2104.07659/) (2021)
- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [Few-Shot Adaptive Gaze Estimation](https://mlsanity.com/paper/1905.01941/) (2019)
- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [A Coarse-to-Fine Adaptive Network for Appearance-Based Gaze Estimation](https://mlsanity.com/paper/2001.00187/) (2020)
- [Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild](https://mlsanity.com/paper/2003.07581/) (2020)
- [CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wild](https://mlsanity.com/paper/2011.14679/) (2020)
- [img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation](https://mlsanity.com/paper/2012.07791/) (2020)

## Built on by (most cited first, 11 of 11)

- [Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels](https://mlsanity.com/paper/2502.20249/) (2025)
- [UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training](https://mlsanity.com/paper/2502.02307/) (2025)
- [OmniGaze: Reward-inspired Generalizable Gaze Estimation In The Wild](https://mlsanity.com/paper/2510.13660/) (2025)
- [For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives](https://mlsanity.com/paper/2407.03268/) (2024)
- [Gaze Estimation for Human-Robot Interaction: Analysis Using the NICO Platform](https://mlsanity.com/paper/2509.24001/) (2025)
- [A Generalized Label Shift Perspective for Cross-Domain Gaze Estimation](https://mlsanity.com/paper/2505.13043/) (2025)
- [Hybrid-Domain Adaptative Representation Learning for Gaze Estimation](https://mlsanity.com/paper/2511.13222/) (2025)
- [GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction](https://mlsanity.com/paper/2605.22359/) (2026)
- [Factor-Informed Uncertainty Distillation for Gaze Estimation](https://mlsanity.com/paper/2607.20072/) (2026)
- [Context-Aware Causal Gaze Forecasting for Human-Vehicle Interaction During In-Cabin Tracking Dropouts](https://mlsanity.com/paper/2609.12374/) (2026)
- [SeetaPsych v1.0: An Open-source Computer Vision Toolkit for Behavior-based Psychological Measurement](https://mlsanity.com/paper/2609.19719/) (2026)

## Abstract (arXiv)

Developing gaze estimation models that generalize well to unseen domains and in-the-wild conditions remains a challenge with no known best solution. This is mostly due to the difficulty of acquiring ground truth data that cover the distribution of faces, head poses, and environments that exist in the real world. Most recent methods attempt to close the gap between specific source and target domains using domain adaptation. In this work, we propose to train general gaze estimation models which can be directly employed in novel environments without adaptation. To do so, we leverage the observation that head, body, and hand pose estimation benefit from revising them as dense 3D coordinate prediction, and similarly express gaze estimation as regression of dense 3D eye meshes. To close the gap between image domains, we create a large-scale dataset of diverse faces with gaze pseudo-annotations, which we extract based on the 3D geometry of the scene, and design a multi-view supervision framework to balance their effect during training. We test our method in the task of gaze generalization, in which we demonstrate improvement of up to 30% compared to state-of-the-art when no ground truth data are available, and up to 10% when they are. The project material are available for research purposes at https://github.com/Vagver/3DGazeNet.