# GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction

arXiv: https://arxiv.org/abs/2605.22359 · HTML: https://mlsanity.com/paper/2605.22359/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 50 of its 90 references that are in the map.
- First posted on arXiv 21 May 2026.

## Builds on (most cited first, 30 of 50)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [Generative Novel View Synthesis with 3D-Aware Diffusion Models](https://mlsanity.com/paper/2304.02602/) (2023)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [3D Gaussian Splatting as Markov Chain Monte Carlo](https://mlsanity.com/paper/2404.09591/) (2024)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2312.03704/) (2023)
- [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [NerfAcc: Efficient Sampling Accelerates NeRFs](https://mlsanity.com/paper/2305.04966/) (2023)
- [3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting](https://mlsanity.com/paper/2412.12507/) (2024)
- [Few-Shot Adaptive Gaze Estimation](https://mlsanity.com/paper/1905.01941/) (2019)
- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Combining 3D Morphable Models: A Large scale Face-and-Head Model](https://mlsanity.com/paper/1903.03785/) (2019)
- [GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation](https://mlsanity.com/paper/2312.01632/) (2023)
- [Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings](https://mlsanity.com/paper/1805.04771/) (2018)

## Abstract (arXiv)

Eye tracking (ET) is a foundational technology for advanced AR/VR applications. However, training ET models for every new ET device is challenging: real data collection is costly and time-consuming, while existing synthetic data generation methods lack realism. To remove the need for additional data collection while maintaining data quality, we introduce a data-driven 3D prior that models the distribution of human eyes across diverse identities, gaze directions, and light settings. This model, which we coin GazePrior, then enables sparse-input 3D reconstruction of annotated data collected with previous ET devices, which can in turn be rendered from the cameras of any target ET device. Our approach synthesizes data with the realism, diversity and ground-truth accuracy of real data collection without its prohibitive costs. Our experiments demonstrate that ET models trained with our synthesized data outperform previous zero-shot methods, achieving higher accuracy and robustness.