# 3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

arXiv: https://arxiv.org/abs/2308.11771 · HTML: https://mlsanity.com/paper/2308.11771/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [Real-Time Face & Eye Tracking and Blink Detection using Event Cameras](https://mlsanity.com/paper/2010.08278/) (2020)
- [Exploiting Spatial Sparsity for Event Cameras with Visual Transformers](https://mlsanity.com/paper/2202.05054/) (2022)
- [Real-Time Gaze Tracking with Event-Driven Eye Segmentation](https://mlsanity.com/paper/2201.07367/) (2022)
- [Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity](https://mlsanity.com/paper/2108.02297/) (2021)
- [Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments](https://mlsanity.com/paper/1511.05768/) (2015)
- [Delta Networks for Optimized Recurrent Network Computation](https://mlsanity.com/paper/1612.05571/) (2016)

## Built on by (most cited first, 30 of 30)

- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware](https://mlsanity.com/paper/2312.00425/) (2023)
- [Event-Based Eye Tracking. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.11770/) (2024)
- [Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba](https://mlsanity.com/paper/2405.06116/) (2024)
- [MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking](https://mlsanity.com/paper/2404.12083/) (2024)
- [Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN](https://mlsanity.com/paper/2404.14279/) (2024)
- [FACET: Fast and Accurate Event-Based Eye Tracking Using Ellipse Modeling for Extended Reality](https://mlsanity.com/paper/2409.15584/) (2024)
- [FAPNet: An Effective Frequency Adaptive Point-based Eye Tracker](https://mlsanity.com/paper/2406.03177/) (2024)
- [Exploring Temporal Dynamics in Event-based Eye Tracker](https://mlsanity.com/paper/2503.23725/) (2025)
- [Event-Based Eye Tracking. 2025 Event-based Vision Workshop](https://mlsanity.com/paper/2504.18249/) (2025)
- [Modeling State Shifting via Local-Global Distillation for Event-Frame Gaze Tracking](https://mlsanity.com/paper/2404.00548/) (2024)
- [EgoEvGesture: Gesture Recognition Based on Egocentric Event Camera](https://mlsanity.com/paper/2503.12419/) (2025)
- [Evaluating Image-Based Face and Eye Tracking with Event Cameras](https://mlsanity.com/paper/2408.10395/) (2024)
- [GG-SSMs: Graph-Generating State Space Models](https://mlsanity.com/paper/2412.12423/) (2024)
- [GazeSCRNN: Event-based Near-eye Gaze Tracking using a Spiking Neural Network](https://mlsanity.com/paper/2503.16012/) (2025)
- [Dual-Path Enhancements in Event-Based Eye Tracking: Augmented Robustness and Adaptive Temporal Modeling](https://mlsanity.com/paper/2504.09960/) (2025)
- [Eye Gaze Tells You Where to Compute: Gaze-Driven Efficient VLMs](https://mlsanity.com/paper/2509.16476/) (2025)
- [Text-to-Events: Synthetic Event Camera Streams from Conditional Text Input](https://mlsanity.com/paper/2406.03439/) (2024)
- [EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing](https://mlsanity.com/paper/2409.18813/) (2024)
- [JaneEye: A 12-nm 2K-FPS 18.9-μJ/Frame Event-based Eye Tracking Accelerator](https://mlsanity.com/paper/2510.01213/) (2025)
- [SynSacc: A Blender-to-V2E Pipeline for Synthetic Neuromorphic Eye-Movement Data and Sim-to-Real Spiking Model Training](https://mlsanity.com/paper/2602.08726/) (2026)
- [LAND: A Longitudinal Analysis of Neuromorphic Datasets](https://mlsanity.com/paper/2602.15973/) (2026)
- [Inference-Time Gaze Refinement for Micro-Expression Recognition: Enhancing Event-Based Eye Tracking with Motion-Aware Post-Processing](https://mlsanity.com/paper/2506.12524/) (2025)
- [Realizing Fully-Integrated, Low-Power, Event-Based Pupil Tracking with Neuromorphic Hardware](https://mlsanity.com/paper/2511.20175/) (2025)
- [Neuromorphic Eye Tracking for Low-Latency Pupil Detection](https://mlsanity.com/paper/2512.09969/) (2025)
- [Reconstruction as a Bridge for Event-Based Visual Question Answering](https://mlsanity.com/paper/2512.11510/) (2025)
- [Event-based high temporal resolution measurement of shock wave motion field](https://mlsanity.com/paper/2512.22474/) (2025)
- [Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling](https://mlsanity.com/paper/2603.14077/) (2026)
- [Event-based Liveness Detection using Temporal Ocular Dynamics: An Exploratory Approach](https://mlsanity.com/paper/2604.26285/) (2026)
- [EveLoad: Cognitive Workload Recognition from Event-Based Eye Movements](https://mlsanity.com/paper/2606.25177/) (2026)

## Abstract (arXiv)

This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a textttv2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at urlhttps://github.com/qinche106/cb-convlstm-eyetracking.