# Event-aided Direct Sparse Odometry

arXiv: https://arxiv.org/abs/2204.07640 · HTML: https://mlsanity.com/paper/2204.07640/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 69 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Event-based Stereo Visual Odometry](https://mlsanity.com/paper/2007.15548/) (2020)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Continuous-time Intensity Estimation Using Event Cameras](https://mlsanity.com/paper/1811.00386/) (2018)
- [Semi-Dense 3D Reconstruction with a Stereo Event Camera](https://mlsanity.com/paper/1807.07429/) (2018)
- [Event-based, 6-DOF Camera Tracking from Photometric Depth Maps](https://mlsanity.com/paper/1607.03468/) (2016)
- [How to Calibrate Your Event Camera](https://mlsanity.com/paper/2105.12362/) (2021)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [FutureMapping: The Computational Structure of Spatial AI Systems](https://mlsanity.com/paper/1803.11288/) (2018)
- [Event-based Camera Pose Tracking using a Generative Event Model](https://mlsanity.com/paper/1510.01972/) (2015)
- [High Speed Event Camera TRacking](https://mlsanity.com/paper/2010.02771/) (2020)
- [Real-Time Panoramic Tracking for Event Cameras](https://mlsanity.com/paper/1703.05161/) (2017)

## Built on by (most cited first, 30 of 92)

- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [E-NeRF: Neural Radiance Fields from a Moving Event Camera](https://mlsanity.com/paper/2208.11300/) (2022)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Ev-NeRF: Event Based Neural Radiance Field](https://mlsanity.com/paper/2206.12455/) (2022)
- [Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion](https://mlsanity.com/paper/2309.08596/) (2023)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [Deformable Neural Radiance Fields using RGB and Event Cameras](https://mlsanity.com/paper/2309.08416/) (2023)
- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion](https://mlsanity.com/paper/2207.10494/) (2022)
- [Generalizing Event-Based Motion Deblurring in Real-World Scenarios](https://mlsanity.com/paper/2308.05932/) (2023)
- [Event-based Stereo Depth Estimation: A Survey](https://mlsanity.com/paper/2409.17680/) (2024)
- [Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering](https://mlsanity.com/paper/2407.11343/) (2024)
- [Event-based Simultaneous Localization and Mapping: A Comprehensive Survey](https://mlsanity.com/paper/2304.09793/) (2023)
- [Microsaccade-inspired Event Camera for Robotics](https://mlsanity.com/paper/2405.17769/) (2024)
- [Implicit Event-RGBD Neural SLAM](https://mlsanity.com/paper/2311.11013/) (2023)
- [Event-based Continuous Color Video Decompression from Single Frames](https://mlsanity.com/paper/2312.00113/) (2023)
- [BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream](https://mlsanity.com/paper/2407.02174/) (2024)
- [EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting](https://mlsanity.com/paper/2410.15392/) (2024)
- [Deep Visual Odometry with Events and Frames](https://mlsanity.com/paper/2309.09947/) (2023)
- [A 5-Point Minimal Solver for Event Camera Relative Motion Estimation](https://mlsanity.com/paper/2309.17054/) (2023)
- [BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Events](https://mlsanity.com/paper/2410.20451/) (2024)
- [Data-Driven Feature Tracking for Event Cameras With and Without Frames](https://mlsanity.com/paper/2211.12826/) (2022)
- [EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering](https://mlsanity.com/paper/2412.07293/) (2024)
- [Event-based Background-Oriented Schlieren](https://mlsanity.com/paper/2311.00434/) (2023)
- [An N-Point Linear Solver for Line and Motion Estimation with Event Cameras](https://mlsanity.com/paper/2404.00842/) (2024)
- [SoDaCam: Software-defined Cameras via Single-Photon Imaging](https://mlsanity.com/paper/2309.00066/) (2023)
- [Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions](https://mlsanity.com/paper/2409.17988/) (2024)
- [EventFly: Event Camera Perception from Ground to the Sky](https://mlsanity.com/paper/2503.19916/) (2025)
- [Real-time 6-DoF Pose Estimation by an Event-based Camera using Active LED Markers](https://mlsanity.com/paper/2310.16618/) (2023)
- [EVI-SAM: Robust, Real-time, Tightly-coupled Event-Visual-Inertial State Estimation and 3D Dense Mapping](https://mlsanity.com/paper/2312.11911/) (2023)

## Abstract (arXiv)

We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time between frames. The method formulates a direct probabilistic approach of observed brightness increments. Per-pixel brightness increments are predicted using a sparse number of selected 3D points and are compared to the events via the brightness increment error to estimate camera motion. The method recovers a semi-dense 3D map using photometric bundle adjustment. EDS is the first method to perform 6-DOF VO using events and frames with a direct approach. By design, it overcomes the problem of changing appearance in indirect methods. We also show that, for a target error performance, EDS can work at lower frame rates than state-of-the-art frame-based VO solutions. This opens the door to low-power motion-tracking applications where frames are sparingly triggered "on demand" and our method tracks the motion in between. We release code and datasets to the public.