# Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware

arXiv: https://arxiv.org/abs/2312.00425 · HTML: https://mlsanity.com/paper/2312.00425/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Learning non-maximum suppression](https://mlsanity.com/paper/1705.02950/) (2017)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [AGIL: Learning Attention from Human for Visuomotor Tasks](https://mlsanity.com/paper/1806.03960/) (2018)
- [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023)
- [Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications](https://mlsanity.com/paper/1901.10906/) (2019)
- [Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues](https://mlsanity.com/paper/1805.03064/) (2018)
- [TinyTracker: Ultra-Fast and Ultra-Low-Power Edge Vision In-Sensor for Gaze Estimation](https://mlsanity.com/paper/2307.07813/) (2023)
- [Sub-mW Neuromorphic SNN audio processing applications with Rockpool and Xylo](https://mlsanity.com/paper/2208.12991/) (2022)
- [Neuromorphic Optical Flow and Real-time Implementation with Event Cameras](https://mlsanity.com/paper/2304.07139/) (2023)
- [WaveSense: Efficient Temporal Convolutions with Spiking Neural Networks for Keyword Spotting](https://mlsanity.com/paper/2111.01456/) (2021)

## Built on by (most cited first, 27 of 27)

- [FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality](https://mlsanity.com/paper/2409.15584/) (2024)
- [TinierHAR: Towards Ultra-Lightweight Deep Learning Models for Efficient Human Activity Recognition on Edge Devices](https://mlsanity.com/paper/2507.07949/) (2025)
- [Exploring Temporal Dynamics in Event-based Eye Tracker](https://mlsanity.com/paper/2503.23725/) (2025)
- [Event-Based Eye Tracking. 2025 Event-based Vision Workshop](https://mlsanity.com/paper/2504.18249/) (2025)
- [PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking Services](https://mlsanity.com/paper/2408.00950/) (2024)
- [Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review](https://mlsanity.com/paper/2603.08725/) (2026)
- [On-device Learning of EEGNet-based Network For Wearable Motor Imagery Brain-Computer Interface](https://mlsanity.com/paper/2409.00083/) (2024)
- [GG-SSMs: Graph-Generating State Space Models](https://mlsanity.com/paper/2412.12423/) (2024)
- [GazeSCRNN: Event-based Near-eye Gaze Tracking using a Spiking Neural Network](https://mlsanity.com/paper/2503.16012/) (2025)
- [Towards Low-Latency Event-based Obstacle Avoidance on a FPGA-Drone](https://mlsanity.com/paper/2504.10400/) (2025)
- [EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing](https://mlsanity.com/paper/2409.18813/) (2024)
- [High-frequency near-eye ground truth for event-based eye tracking](https://mlsanity.com/paper/2502.03057/) (2025)
- [RGB-Event Fusion with Self-Attention for Collision Prediction](https://mlsanity.com/paper/2505.04258/) (2025)
- [JaneEye: A 12-nm 2K-FPS 18.9-μJ/Frame Event-based Eye Tracking Accelerator](https://mlsanity.com/paper/2510.01213/) (2025)
- [SynSacc: A Blender-to-V2E Pipeline for Synthetic Neuromorphic Eye-Movement Data and Sim-to-Real Spiking Model Training](https://mlsanity.com/paper/2602.08726/) (2026)
- [EETnet: a CNN for Gaze Detection and Tracking for Smart-Eyewear](https://mlsanity.com/paper/2511.04779/) (2025)
- [Realizing Fully-Integrated, Low-Power, Event-Based Pupil Tracking with Neuromorphic Hardware](https://mlsanity.com/paper/2511.20175/) (2025)
- [Neuromorphic Eye Tracking for Low-Latency Pupil Detection](https://mlsanity.com/paper/2512.09969/) (2025)
- [Event-based high temporal resolution measurement of shock wave motion field](https://mlsanity.com/paper/2512.22474/) (2025)
- [Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling](https://mlsanity.com/paper/2603.14077/) (2026)
- [Event-based Liveness Detection using Temporal Ocular Dynamics: An Exploratory Approach](https://mlsanity.com/paper/2604.26285/) (2026)
- [Low Latency Gaze Tracking via Latent Optical Sensing](https://mlsanity.com/paper/2605.17990/) (2026)
- [Event-Based Vision in Space: Applications, Trends, and Future Directions](https://mlsanity.com/paper/2606.01280/) (2026)
- [EveLoad: Cognitive Workload Recognition from Event-Based Eye Movements](https://mlsanity.com/paper/2606.25177/) (2026)
- [Event-Based Motion Estimation via Oriented Distance Fields](https://mlsanity.com/paper/2608.24223/) (2026)
- [EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition](https://mlsanity.com/paper/2609.14498/) (2026)
- [You Don't Need To Train: Agentic Heuristic Learning Studio for Executable Human Activity Recognition](https://mlsanity.com/paper/2609.16065/) (2026)

## Abstract (arXiv)

This paper introduces a neuromorphic methodology for eye tracking, harnessing pure event data captured by a Dynamic Vision Sensor (DVS) camera. The framework integrates a directly trained Spiking Neuron Network (SNN) regression model and leverages a state-of-the-art low power edge neuromorphic processor - Speck, collectively aiming to advance the precision and efficiency of eye-tracking systems. First, we introduce a representative event-based eye-tracking dataset, "Ini-30", which was collected with two glass-mounted DVS cameras from thirty volunteers. Then,a SNN model, based on Integrate And Fire (IAF) neurons, named "Retina", is described , featuring only 64k parameters (6.63x fewer than the latest) and achieving pupil tracking error of only 3.24 pixels in a 64x64 DVS input. The continous regression output is obtained by means of convolution using a non-spiking temporal 1D filter slided across the output spiking layer. Finally, we evaluate Retina on the neuromorphic processor, showing an end-to-end power between 2.89-4.8 mW and a latency of 5.57-8.01 mS dependent on the time window. We also benchmark our model against the latest event-based eye-tracking method, "3ET", which was built upon event frames. Results show that Retina achieves superior precision with 1.24px less pupil centroid error and reduced computational complexity with 35 times fewer MAC operations. We hope this work will open avenues for further investigation of close-loop neuromorphic solutions and true event-based training pursuing edge performance.