# Continuous-Time Gaussian Process Motion-Compensation for Event-vision Pattern Tracking with Distance Fields

arXiv: https://arxiv.org/abs/2303.02672 · HTML: https://mlsanity.com/paper/2303.02672/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 67 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Event-aided Direct Sparse Odometry](https://mlsanity.com/paper/2204.07640/) (2022)
- [Speed Invariant Time Surface for Learning to Detect Corner Points with Event-Based Cameras](https://mlsanity.com/paper/1903.11332/) (2019)
- [Exploring Event Camera-based Odometry for Planetary Robots](https://mlsanity.com/paper/2204.05880/) (2022)
- [Globally Optimal Contrast Maximisation for Event-based Motion Estimation](https://mlsanity.com/paper/2002.10686/) (2020)
- [IDOL: A Framework for IMU-DVS Odometry using Lines](https://mlsanity.com/paper/2008.05749/) (2020)
- [FA-Harris: A Fast and Asynchronous Corner Detector for Event Cameras](https://mlsanity.com/paper/1906.10925/) (2019)
- [Event-based Camera Pose Tracking using a Generative Event Model](https://mlsanity.com/paper/1510.01972/) (2015)
- [Comparing Representations in Tracking for Event Camera-based SLAM](https://mlsanity.com/paper/2104.09887/) (2021)
- [High Speed Event Camera TRacking](https://mlsanity.com/paper/2010.02771/) (2020)
- [LuvHarris: A Practical Corner Detector for Event-cameras](https://mlsanity.com/paper/2105.11443/) (2021)
- [Asynchronous Spatial Image Convolutions for Event Cameras](https://mlsanity.com/paper/1812.00438/) (2018)
- [Continuous Event-Line Constraint for Closed-Form Velocity Initialization](https://mlsanity.com/paper/2109.04313/) (2021)
- [Globally-Optimal Event Camera Motion Estimation](https://mlsanity.com/paper/2203.03914/) (2022)
- [Asynchronous Optimisation for Event-based Visual Odometry](https://mlsanity.com/paper/2203.01037/) (2022)
- [Batch Nonlinear Continuous-Time Trajectory Estimation as Exactly Sparse Gaussian Process Regression](https://mlsanity.com/paper/1412.0630/) (2014)
- [Event-Based Feature Tracking in Continuous Time with Sliding Window Optimization](https://mlsanity.com/paper/2107.04536/) (2021)
- [Faithful Euclidean Distance Field from Log-Gaussian Process Implicit Surfaces](https://mlsanity.com/paper/2010.11487/) (2020)
- [Hough2Map -- Iterative Event-based Hough Transform for High-Speed Railway Mapping](https://mlsanity.com/paper/2102.08145/) (2021)
- [Accurate Gaussian-Process-based Distance Fields with applications to Echolocation and Mapping](https://mlsanity.com/paper/2302.13005/) (2023)
- [Event-Based Features Selection and Tracking from Intertwined Estimation of Velocity and Generative Contours](https://mlsanity.com/paper/1811.07839/) (2018)

## Built on by (most cited first, 2 of 2)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Mixing Data-driven and Geometric Models for Satellite Docking Port State Estimation using an RGB or Event Camera](https://mlsanity.com/paper/2409.15581/) (2024)

## Abstract (arXiv)

This work addresses the issue of motion compensation and pattern tracking in event camera data. An event camera generates asynchronous streams of events triggered independently by each of the pixels upon changes in the observed intensity. Providing great advantages in low-light and rapid-motion scenarios, such unconventional data present significant research challenges as traditional vision algorithms are not directly applicable to this sensing modality. The proposed method decomposes the tracking problem into a local SE(2) motion-compensation step followed by a homography registration of small motion-compensated event batches. The first component relies on Gaussian Process (GP) theory to model the continuous occupancy field of the events in the image plane and embed the camera trajectory in the covariance kernel function. In doing so, estimating the trajectory is done similarly to GP hyperparameter learning by maximising the log marginal likelihood of the data. The continuous occupancy fields are turned into distance fields and used as templates for homography-based registration. By benchmarking the proposed method against other state-of-the-art techniques, we show that our open-source implementation performs high-accuracy motion compensation and produces high-quality tracks in real-world scenarios.