# EVIT: Event-based Visual-Inertial Tracking in Semi-Dense Maps Using Windowed Nonlinear Optimization

arXiv: https://arxiv.org/abs/2408.01370 · HTML: https://mlsanity.com/paper/2408.01370/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 19 of 19)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Event-based Stereo Visual Odometry](https://mlsanity.com/paper/2007.15548/) (2020)
- [EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera](https://mlsanity.com/paper/1908.11505/) (2019)
- [Event-based, 6-DOF Camera Tracking from Photometric Depth Maps](https://mlsanity.com/paper/1607.03468/) (2016)
- [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020)
- [EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream](https://mlsanity.com/paper/2012.06475/) (2020)
- [VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM](https://mlsanity.com/paper/2207.01404/) (2022)
- [DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions](https://mlsanity.com/paper/2202.02556/) (2022)
- [IDOL: A Framework for IMU-DVS Odometry using Lines](https://mlsanity.com/paper/2008.05749/) (2020)
- [High Speed Event Camera TRacking](https://mlsanity.com/paper/2010.02771/) (2020)
- [A 5-Point Minimal Solver for Event Camera Relative Motion Estimation](https://mlsanity.com/paper/2309.17054/) (2023)
- [Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment](https://mlsanity.com/paper/2012.08228/) (2020)
- [Continuous Event-Line Constraint for Closed-Form Velocity Initialization](https://mlsanity.com/paper/2109.04313/) (2021)
- [Cross-Modal Semi-Dense 6-DoF Tracking of an Event Camera in Challenging Conditions](https://mlsanity.com/paper/2401.08043/) (2024)
- [Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environment](https://mlsanity.com/paper/2301.06648/) (2023)
- [Tight Fusion of Events and Inertial Measurements for Direct Velocity Estimation](https://mlsanity.com/paper/2401.09296/) (2024)

## Built on by (most cited first, 3 of 3)

- [GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting](https://mlsanity.com/paper/2409.19228/) (2024)
- [E2EGS: Event-to-Edge Gaussian Splatting for Pose-Free 3D Reconstruction](https://mlsanity.com/paper/2603.14684/) (2026)
- [LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization](https://mlsanity.com/paper/2603.01839/) (2026)

## Abstract (arXiv)

Event cameras are an interesting visual exteroceptive sensor that reacts to brightness changes rather than integrating absolute image intensities. Owing to this design, the sensor exhibits strong performance in situations of challenging dynamics and illumination conditions. While event-based simultaneous tracking and mapping remains a challenging problem, a number of recent works have pointed out the sensor's suitability for prior map-based tracking. By making use of cross-modal registration paradigms, the camera's ego-motion can be tracked across a large spectrum of illumination and dynamics conditions on top of accurate maps that have been created a priori by more traditional sensors. The present paper follows up on a recently introduced event-based geometric semi-dense tracking paradigm, and proposes the addition of inertial signals in order to robustify the estimation. More specifically, the added signals provide strong cues for pose initialization as well as regularization during windowed, multi-frame tracking. As a result, the proposed framework achieves increased performance under challenging illumination conditions as well as a reduction of the rate at which intermediate event representations need to be registered in order to maintain stable tracking across highly dynamic sequences. Our evaluation focuses on a diverse set of real world sequences and comprises a comparison of our proposed method against a purely event-based alternative running at different rates.