# Fusing Event-based Camera and Radar for SLAM Using Spiking Neural Networks with Continual STDP Learning

arXiv: https://arxiv.org/abs/2210.04236 · HTML: https://mlsanity.com/paper/2210.04236/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 9 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [RADIATE: A Radar Dataset for Automotive Perception in Bad Weather](https://mlsanity.com/paper/2010.09076/) (2020)
- [Deep Learning without Weight Transport](https://mlsanity.com/paper/1904.05391/) (2019)
- [Event-driven Vision and Control for UAVs on a Neuromorphic Chip](https://mlsanity.com/paper/2108.03694/) (2021)
- [Spiking Neural Network on Neuromorphic Hardware for Energy-Efficient Unidimensional SLAM](https://mlsanity.com/paper/1903.02504/) (2019)
- [ORBSLAM-Atlas: a robust and accurate multi-map system](https://mlsanity.com/paper/1908.11585/) (2019)
- [Fail-Safe Human Detection for Drones Using a Multi-Modal Curriculum Learning Approach](https://mlsanity.com/paper/2109.13666/) (2021)
- [LatentSLAM: unsupervised multi-sensor representation learning for localization and mapping](https://mlsanity.com/paper/2105.03265/) (2021)
- [Obstacle Avoidance onboard MAVs using a FMCW RADAR](https://mlsanity.com/paper/2103.02050/) (2021)
- [A Low-Complexity Radar Detector Outperforming OS-CFAR for Indoor Drone Obstacle Avoidance](https://mlsanity.com/paper/2107.07250/) (2021)
- [A New Look at Spike-Timing-Dependent Plasticity Networks for Spatio-Temporal Feature Learning](https://mlsanity.com/paper/2111.00791/) (2021)

## Built on by (most cited first, 13 of 13)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Event-based Simultaneous Localization and Mapping: A Comprehensive Survey](https://mlsanity.com/paper/2304.09793/) (2023)
- [Applications of Spiking Neural Networks in Visual Place Recognition](https://mlsanity.com/paper/2311.13186/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [Embodied Neuromorphic Artificial Intelligence for Robotics: Perspectives, Challenges, and Research Development Stack](https://mlsanity.com/paper/2404.03325/) (2024)
- [Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing](https://mlsanity.com/paper/2502.14992/) (2025)
- [Resource-Efficient Gesture Recognition using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation](https://mlsanity.com/paper/2401.06563/) (2024)
- [Towards Chip-in-the-loop Spiking Neural Network Training via Metropolis-Hastings Sampling](https://mlsanity.com/paper/2402.06284/) (2024)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [ADMM-Based Training for Spiking Neural Networks](https://mlsanity.com/paper/2505.05527/) (2025)
- [SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities](https://mlsanity.com/paper/2507.16151/) (2025)
- [NERVE: A Neuromorphic Vision and Radar Ensemble for Multi-Sensor Fusion Research](https://mlsanity.com/paper/2605.16414/) (2026)

## Abstract (arXiv)

This work proposes a first-of-its-kind SLAM architecture fusing an event-based camera and a Frequency Modulated Continuous Wave (FMCW) radar for drone navigation. Each sensor is processed by a bio-inspired Spiking Neural Network (SNN) with continual Spike-Timing-Dependent Plasticity (STDP) learning, as observed in the brain. In contrast to most learning-based SLAM systems%, which a) require the acquisition of a representative dataset of the environment in which navigation must be performed and b) require an off-line training phase, our method does not require any offline training phase, but rather the SNN continuously learns features from the input data on the fly via STDP. At the same time, the SNN outputs are used as feature descriptors for loop closure detection and map correction. We conduct numerous experiments to benchmark our system against state-of-the-art RGB methods and we demonstrate the robustness of our DVS-Radar SLAM approach under strong lighting variations.