# Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling

arXiv: https://arxiv.org/abs/2603.14077 · HTML: https://mlsanity.com/paper/2603.14077/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 70 references that are in the map.
- First posted on arXiv 14 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Learning Longer-term Dependencies in RNNs with Auxiliary Losses](https://mlsanity.com/paper/1803.00144/) (2018)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [CED: Color Event Camera Dataset](https://mlsanity.com/paper/1904.10772/) (2019)
- [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023)
- [Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware](https://mlsanity.com/paper/2312.00425/) (2023)
- [EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking](https://mlsanity.com/paper/2007.09600/) (2020)
- [Event-Based Eye Tracking. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.11770/) (2024)
- [A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalization](https://mlsanity.com/paper/2403.19412/) (2024)
- [A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera](https://mlsanity.com/paper/2404.08858/) (2024)
- [Bina-Rep Event Frames: a Simple and Effective Representation for Event-based cameras](https://mlsanity.com/paper/2202.13662/) (2022)
- [To Spike or Not To Spike: A Digital Hardware Perspective on Deep Learning Acceleration](https://mlsanity.com/paper/2306.15749/) (2023)
- [R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models](https://mlsanity.com/paper/2409.14216/) (2024)
- [EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking](https://mlsanity.com/paper/2205.01947/) (2022)
- [Exploring Temporal Dynamics in Event-based Eye Tracker](https://mlsanity.com/paper/2503.23725/) (2025)

## Abstract (arXiv)

Eye feature extraction from event-based data streams can be performed efficiently and with low energy consumption, offering great utility to real-world eye tracking pipelines. However, few eye feature extractors are designed to handle sudden changes in event density caused by the changes between gaze behaviors that vary in their kinematics, leading to degraded prediction performance. In this work, we address this problem by introducing the adaptive inference state space model (AISSM), a novel architecture for feature extraction that is capable of dynamically adjusting the relative weight placed on current versus recent information. This relative weighting is determined via estimates of the signal-to-noise ratio and event density produced by a complementary dynamic confidence network. Lastly, we craft and evaluate a novel learning technique that improves training efficiency. Experimental results demonstrate that the AISSM system outperforms state-of-the-art models for event-based eye feature extraction.