# FAPNet: An Effective Frequency Adaptive Point-based Eye Tracker

arXiv: https://arxiv.org/abs/2406.03177 · HTML: https://mlsanity.com/paper/2406.03177/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling](https://mlsanity.com/paper/1904.03375/) (2019)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [Unifying Motion Deblurring and Frame Interpolation with Events](https://mlsanity.com/paper/2203.12178/) (2022)
- [Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2202.13589/) (2022)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [Temporal Binary Representation for Event-Based Action Recognition](https://mlsanity.com/paper/2010.08946/) (2020)
- [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023)
- [SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition](https://mlsanity.com/paper/2310.07189/) (2023)
- [Real-Time Face & Eye Tracking and Blink Detection using Event Cameras](https://mlsanity.com/paper/2010.08278/) (2020)
- [Real-Time Gaze Tracking with Event-Driven Eye Segmentation](https://mlsanity.com/paper/2201.07367/) (2022)
- [CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2402.04663/) (2024)
- [Gaze-Contingent Ocular Parallax Rendering for Virtual Reality](https://mlsanity.com/paper/1906.09740/) (2019)
- [Event-Based Eye Tracking. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.11770/) (2024)
- [Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba](https://mlsanity.com/paper/2405.06116/) (2024)
- [A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalization](https://mlsanity.com/paper/2403.19412/) (2024)
- [DA4Event: towards bridging the Sim-to-Real Gap for Event Cameras using Domain Adaptation](https://mlsanity.com/paper/2103.12768/) (2021)
- [TTPOINT: A Tensorized Point Cloud Network for Lightweight Action Recognition with Event Cameras](https://mlsanity.com/paper/2308.09993/) (2023)
- [AFPR-CIM: An Analog-Domain Floating-Point RRAM-based Compute-In-Memory Architecture with Dynamic Range Adaptive FP-ADC](https://mlsanity.com/paper/2402.13798/) (2024)

## Built on by (most cited first, 6 of 6)

- [FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality](https://mlsanity.com/paper/2409.15584/) (2024)
- [Exploring Temporal Dynamics in Event-based Eye Tracker](https://mlsanity.com/paper/2503.23725/) (2025)
- [Event-Based Eye Tracking. 2025 Event-based Vision Workshop](https://mlsanity.com/paper/2504.18249/) (2025)
- [Event-based Motion Deblurring via Multi-Temporal Granularity Fusion](https://mlsanity.com/paper/2412.11866/) (2024)
- [DRRNet: Macro-Micro Feature Fusion and Dual Reverse Refinement for Camouflaged Object Detection](https://mlsanity.com/paper/2505.09168/) (2025)
- [EveLoad: Cognitive Workload Recognition from Event-Based Eye Movements](https://mlsanity.com/paper/2606.25177/) (2026)

## Abstract (arXiv)

Eye tracking is crucial for human-computer interaction in different domains. Conventional cameras encounter challenges such as power consumption and image quality during different eye movements, prompting the need for advanced solutions with ultra-fast, low-power, and accurate eye trackers. Event cameras, fundamentally designed to capture information about moving objects, exhibit low power consumption and high temporal resolution. This positions them as an alternative to traditional cameras in the realm of eye tracking. Nevertheless, existing event-based eye tracking networks neglect the pivotal sparse and fine-grained temporal information in events, resulting in unsatisfactory performance. Moreover, the energy-efficient features are further compromised by the use of excessively complex models, hindering efficient deployment on edge devices. In this paper, we utilize Point Cloud as the event representation to harness the high temporal resolution and sparse characteristics of events in eye tracking tasks. We rethink the point-based architecture PEPNet with preprocessing the long-term relationships between samples, leading to the innovative design of FAPNet. A frequency adaptive mechanism is designed to realize adaptive tracking according to the speed of the pupil movement and the Inter Sample LSTM module is introduced to utilize the temporal correlation between samples. In the Event-based Eye Tracking Challenge, we utilize vanilla PEPNet, which is the former work to achieve the p10 accuracy of 97.95%. On the SEET synthetic dataset, FAPNet can achieve state-of-the-art while consuming merely 10% of the PEPNet's computational resources. Notably, the computational demand of FAPNet is independent of the sensor's spatial resolution, enhancing its applicability on resource-limited edge devices.