# Modeling State Shifting via Local-Global Distillation for Event-Frame Gaze Tracking

arXiv: https://arxiv.org/abs/2404.00548 · HTML: https://mlsanity.com/paper/2404.00548/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 94 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification](https://mlsanity.com/paper/2001.01536/) (2020)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline](https://mlsanity.com/paper/2309.14611/) (2023)
- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://mlsanity.com/paper/2211.11010/) (2022)
- [Appearance-Based Gaze Estimation Using Dilated-Convolutions](https://mlsanity.com/paper/1903.07296/) (2019)
- [Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction](https://mlsanity.com/paper/1405.0006/) (2014)
- [Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers](https://mlsanity.com/paper/2307.04129/) (2023)
- [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023)
- [Real-Time Face & Eye Tracking and Blink Detection using Event Cameras](https://mlsanity.com/paper/2010.08278/) (2020)
- [Real-Time Gaze Tracking with Event-Driven Eye Segmentation](https://mlsanity.com/paper/2201.07367/) (2022)
- [DVGaze: Dual-View Gaze Estimation](https://mlsanity.com/paper/2308.10310/) (2023)
- [Light-weight Head Pose Invariant Gaze Tracking](https://mlsanity.com/paper/1804.08572/) (2018)
- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Benefits of temporal information for appearance-based gaze estimation](https://mlsanity.com/paper/2005.11670/) (2020)
- [The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation](https://mlsanity.com/paper/2106.14183/) (2021)
- [Continuous Gaze Tracking With Implicit Saliency-Aware Calibration on Mobile Devices](https://mlsanity.com/paper/2209.15196/) (2022)
- [3D Gaze Estimation from 2D Pupil Positions on Monocular Head-Mounted Eye Trackers](https://mlsanity.com/paper/1601.02644/) (2016)
- [JEDI: Joint Expert Distillation in a Semi-Supervised Multi-Dataset Student-Teacher Scenario for Video Action Recognition](https://mlsanity.com/paper/2308.04934/) (2023)

## Built on by (most cited first, 4 of 4)

- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [Dark-EvGS: Event Camera as an Eye for Radiance Field in the Dark](https://mlsanity.com/paper/2507.11931/) (2025)
- [ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation](https://mlsanity.com/paper/2412.09105/) (2024)
- [UniE2F: A Unified Diffusion Framework for Event-to-Frame Reconstruction with Video Foundation Models](https://mlsanity.com/paper/2602.19202/) (2026)

## Abstract (arXiv)

This paper tackles the problem of passive gaze estimation using both event and frame data. Considering the inherently different physiological structures, it is intractable to accurately estimate gaze purely based on a given state. Thus, we reformulate gaze estimation as the quantification of the state shifting from the current state to several prior registered anchor states. Specifically, we propose a two-stage learning-based gaze estimation framework that divides the whole gaze estimation process into a coarse-to-fine approach involving anchor state selection and final gaze location. Moreover, to improve the generalization ability, instead of learning a large gaze estimation network directly, we align a group of local experts with a student network, where a novel denoising distillation algorithm is introduced to utilize denoising diffusion techniques to iteratively remove inherent noise in event data. Extensive experiments demonstrate the effectiveness of the proposed method, which surpasses state-of-the-art methods by a large margin of 15%. The code will be publicly available at https://github.com/jdjdli/DenoisedistillEFgazetracker.