# End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking

arXiv: https://arxiv.org/abs/2002.05911 · HTML: https://mlsanity.com/paper/2002.05911/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 21 of 21)

- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Object Tracking by Jointly Exploiting Frame and Event Domain](https://mlsanity.com/paper/2109.09052/) (2021)
- [Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://mlsanity.com/paper/2211.11010/) (2022)
- [Hierarchical Neural Memory Network for Low Latency Event Processing](https://mlsanity.com/paper/2305.17852/) (2023)
- [Frame-Event Alignment and Fusion Network for High Frame Rate Tracking](https://mlsanity.com/paper/2305.15688/) (2023)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [Globally-Optimal Contrast Maximisation for Event Cameras](https://mlsanity.com/paper/2206.05127/) (2022)
- [CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Cameras](https://mlsanity.com/paper/2401.02826/) (2024)
- [Asynchronous Blob Tracker for Event Cameras](https://mlsanity.com/paper/2307.10593/) (2023)
- [LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising](https://mlsanity.com/paper/2405.19718/) (2024)
- [Adversarial Attack for Asynchronous Event-based Data](https://mlsanity.com/paper/2112.13534/) (2021)
- [TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking](https://mlsanity.com/paper/2405.05004/) (2024)
- [Distractor-aware Event-based Tracking](https://mlsanity.com/paper/2310.14194/) (2023)
- [Multi-domain Collaborative Feature Representation for Robust Visual Object Tracking](https://mlsanity.com/paper/2108.04521/) (2021)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks](https://mlsanity.com/paper/2109.13456/) (2021)
- [Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera](https://mlsanity.com/paper/2512.21053/) (2025)
- [Tracking-Assisted Object Detection with Event Cameras](https://mlsanity.com/paper/2403.18330/) (2024)
- [EHGCN: Hierarchical Euclidean-Hyperbolic Fusion via Motion-Aware GCN for Hybrid Event Stream Perception](https://mlsanity.com/paper/2504.16616/) (2025)
- [Event Data Association via Robust Model Fitting for Event-based Object Tracking](https://mlsanity.com/paper/2110.12962/) (2021)
- [Optimizing Multi-Modality Trackers via Significance-Regularized Tuning](https://mlsanity.com/paper/2508.17488/) (2025)

## Abstract (arXiv)

Event cameras, which are asynchronous bio-inspired vision sensors, have shown great potential in computer vision and artificial intelligence. However, the application of event cameras to object-level motion estimation or tracking is still in its infancy. The main idea behind this work is to propose a novel deep neural network to learn and regress a parametric object-level motion/transform model for event-based object tracking. To achieve this goal, we propose a synchronous Time-Surface with Linear Time Decay (TSLTD) representation, which effectively encodes the spatio-temporal information of asynchronous retinal events into TSLTD frames with clear motion patterns. We feed the sequence of TSLTD frames to a novel Retinal Motion Regression Network (RMRNet) to perform an end-to-end 5-DoF object motion regression. Our method is compared with state-of-the-art object tracking methods, that are based on conventional cameras or event cameras. The experimental results show the superiority of our method in handling various challenging environments such as fast motion and low illumination conditions.