# A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera

arXiv: https://arxiv.org/abs/2404.08858 · HTML: https://mlsanity.com/paper/2404.08858/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 13 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Tailor: Altering Skip Connections for Resource-Efficient Inference](https://mlsanity.com/paper/2301.07247/) (2023)

## Built on by (most cited first, 14 of 14)

- [Event-Based Eye Tracking. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.11770/) (2024)
- [FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality](https://mlsanity.com/paper/2409.15584/) (2024)
- [Exploring Temporal Dynamics in Event-based Eye Tracker](https://mlsanity.com/paper/2503.23725/) (2025)
- [Event-Based Eye Tracking. 2025 Event-based Vision Workshop](https://mlsanity.com/paper/2504.18249/) (2025)
- [PLEIADES: Building Temporal Kernels with Orthogonal Polynomials](https://mlsanity.com/paper/2405.12179/) (2024)
- [Dual-Path Enhancements in Event-Based Eye Tracking: Augmented Robustness and Adaptive Temporal Modeling](https://mlsanity.com/paper/2504.09960/) (2025)
- [Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learning](https://mlsanity.com/paper/2509.24968/) (2025)
- [JaneEye: A 12-nm 2K-FPS 18.9-μJ/Frame Event-based Eye Tracking Accelerator](https://mlsanity.com/paper/2510.01213/) (2025)
- [aTENNuate: Optimized Real-time Speech Enhancement with Deep SSMs on Raw Audio](https://mlsanity.com/paper/2409.03377/) (2024)
- [Inference-Time Gaze Refinement for Micro-Expression Recognition: Enhancing Event-Based Eye Tracking with Motion-Aware Post-Processing](https://mlsanity.com/paper/2506.12524/) (2025)
- [Realizing Fully-Integrated, Low-Power, Event-Based Pupil Tracking with Neuromorphic Hardware](https://mlsanity.com/paper/2511.20175/) (2025)
- [Neuromorphic Eye Tracking for Low-Latency Pupil Detection](https://mlsanity.com/paper/2512.09969/) (2025)
- [Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling](https://mlsanity.com/paper/2603.14077/) (2026)
- [EveLoad: Cognitive Workload Recognition from Event-Based Eye Movements](https://mlsanity.com/paper/2606.25177/) (2026)

## Abstract (arXiv)

Event-based data are commonly encountered in edge computing environments where efficiency and low latency are critical. To interface with such data and leverage their rich temporal features, we propose a causal spatiotemporal convolutional network. This solution targets efficient implementation on edge-appropriate hardware with limited resources in three ways: 1) deliberately targets a simple architecture and set of operations (convolutions, ReLU activations) 2) can be configured to perform online inference efficiently via buffering of layer outputs 3) can achieve more than 90% activation sparsity through regularization during training, enabling very significant efficiency gains on event-based processors. In addition, we propose a general affine augmentation strategy acting directly on the events, which alleviates the problem of dataset scarcity for event-based systems. We apply our model on the AIS 2024 event-based eye tracking challenge, reaching a score of 0.9916 p10 accuracy on the Kaggle private testset.