# Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios

arXiv: https://arxiv.org/abs/1709.06310 · HTML: https://mlsanity.com/paper/1709.06310/

## Key facts

- Cited by 150 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 19 Sep 2017.

## Built on by (most cited first, 30 of 150)

- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Event-aided Direct Sparse Odometry](https://mlsanity.com/paper/2204.07640/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [ESS: Learning Event-based Semantic Segmentation from Still Images](https://mlsanity.com/paper/2203.10016/) (2022)
- [E-NeRF: Neural Radiance Fields from a Moving Event Camera](https://mlsanity.com/paper/2208.11300/) (2022)
- [How to Calibrate Your Event Camera](https://mlsanity.com/paper/2105.12362/) (2021)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM](https://mlsanity.com/paper/2207.01404/) (2022)
- [Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion](https://mlsanity.com/paper/2309.08596/) (2023)
- [SODFormer: Streaming Object Detection with Transformer Using Events and Frames](https://mlsanity.com/paper/2308.04047/) (2023)
- [DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions](https://mlsanity.com/paper/2202.02556/) (2022)
- [Exploring Event Camera-based Odometry for Planetary Robots](https://mlsanity.com/paper/2204.05880/) (2022)
- [Event Collapse in Contrast Maximization Frameworks](https://mlsanity.com/paper/2207.04007/) (2022)
- [GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge](https://mlsanity.com/paper/2201.07705/) (2022)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization](https://mlsanity.com/paper/2403.08119/) (2024)
- [Mitigating Motion Blur in Neural Radiance Fields with Events and Frames](https://mlsanity.com/paper/2403.19780/) (2024)
- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera](https://mlsanity.com/paper/2408.03225/) (2024)
- [The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data](https://mlsanity.com/paper/2106.06887/) (2021)
- [Comparing Representations in Tracking for Event Camera-based SLAM](https://mlsanity.com/paper/2104.09887/) (2021)
- [M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground Robots](https://mlsanity.com/paper/2112.13659/) (2021)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [ESL: Event-based Structured Light](https://mlsanity.com/paper/2111.15510/) (2021)
- [Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision](https://mlsanity.com/paper/2112.03423/) (2021)
- [Spatiotemporal Registration for Event-based Visual Odometry](https://mlsanity.com/paper/2103.05955/) (2021)
- [PL-EVIO: Robust Monocular Event-based Visual Inertial Odometry with Point and Line Features](https://mlsanity.com/paper/2209.12160/) (2022)
- [LuvHarris: A Practical Corner Detector for Event-cameras](https://mlsanity.com/paper/2105.11443/) (2021)
- [EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset](https://mlsanity.com/paper/2312.08220/) (2023)

## Abstract (arXiv)

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high speed motions or in scenes characterized by high dynamic range. However, event cameras output only little information when the amount of motion is limited, such as in the case of almost still motion. Conversely, standard cameras provide instant and rich information about the environment most of the time (in low-speed and good lighting scenarios), but they fail severely in case of fast motions, or difficult lighting such as high dynamic range or low light scenes. In this paper, we present the first state estimation pipeline that leverages the complementary advantages of these two sensors by fusing in a tightly-coupled manner events, standard frames, and inertial measurements. We show on the publicly available Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement of 130% over event-only pipelines, and 85% over standard-frames-only visual-inertial systems, while still being computationally tractable. Furthermore, we use our pipeline to demonstrate - to the best of our knowledge - the first autonomous quadrotor flight using an event camera for state estimation, unlocking flight scenarios that were not reachable with traditional visual-inertial odometry, such as low-light environments and high-dynamic range scenes.