# Cross-modal Place Recognition in Image Databases using Event-based Sensors

arXiv: https://arxiv.org/abs/2307.01047 · HTML: https://mlsanity.com/paper/2307.01047/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 28 of 28)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Stereo Visual Odometry](https://mlsanity.com/paper/2007.15548/) (2020)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [Continuous-Time Visual-Inertial Odometry for Event Cameras](https://mlsanity.com/paper/1702.07389/) (2017)
- [Event-based, 6-DOF Camera Tracking from Photometric Depth Maps](https://mlsanity.com/paper/1607.03468/) (2016)
- [Event-based visual place recognition with ensembles of temporal windows](https://mlsanity.com/paper/2006.02826/) (2020)
- [DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions](https://mlsanity.com/paper/2202.02556/) (2022)
- [Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Cameras](https://mlsanity.com/paper/2112.10591/) (2021)
- [How Many Events do You Need? Event-based Visual Place Recognition Using Sparse But Varying Pixels](https://mlsanity.com/paper/2206.13673/) (2022)
- [DeepCF: A Unified Framework of Representation Learning and Matching Function Learning in Recommender System](https://mlsanity.com/paper/1901.04704/) (2019)
- [Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments](https://mlsanity.com/paper/2109.06452/) (2021)
- [Visual Odometry with an Event Camera Using Continuous Ray Warping and Volumetric Contrast Maximization](https://mlsanity.com/paper/2107.03011/) (2021)
- [FE-Fusion-VPR: Attention-based Multi-Scale Network Architecture for Visual Place Recognition by Fusing Frames and Events](https://mlsanity.com/paper/2211.12244/) (2022)
- [Asynchronous Optimisation for Event-based Visual Odometry](https://mlsanity.com/paper/2203.01037/) (2022)
- [Deep Architectures and Ensembles for Semantic Video Classification](https://mlsanity.com/paper/1807.01026/) (2018)

## Built on by (most cited first, 1 of 1)

- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)

## Abstract (arXiv)

Visual place recognition is an important problem towards global localization in many robotics tasks. One of the biggest challenges is that it may suffer from illumination or appearance changes in surrounding environments. Event cameras are interesting alternatives to frame-based sensors as their high dynamic range enables robust perception in difficult illumination conditions. However, current event-based place recognition methods only rely on event information, which restricts downstream applications of VPR. In this paper, we present the first cross-modal visual place recognition framework that is capable of retrieving regular images from a database given an event query. Our method demonstrates promising results with respect to the state-of-the-art frame-based and event-based methods on the Brisbane-Event-VPR dataset under different scenarios. We also verify the effectiveness of the combination of retrieval and classification, which can boost performance by a large margin.