# Joint detection and matching of feature points in multimodal images

arXiv: https://arxiv.org/abs/1810.12941 · HTML: https://mlsanity.com/paper/1810.12941/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 10 of 10)

- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Learning multi-domain feature relation for visible and Long-wave Infrared image patch matching](https://mlsanity.com/paper/2308.04880/) (2023)
- [A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration](https://mlsanity.com/paper/2507.18551/) (2025)
- [XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration](https://mlsanity.com/paper/2411.07430/) (2024)
- [Attention-Based Multimodal Image Matching](https://mlsanity.com/paper/2103.11247/) (2021)
- [SA-DNet: A on-demand semantic object registration network adapting to non-rigid deformation](https://mlsanity.com/paper/2210.09900/) (2022)
- [LSGDDN-LCD: An Appearance-based Loop Closure Detection using Local Superpixel Grid Descriptors and Incremental Dynamic Nodes](https://mlsanity.com/paper/2304.03872/) (2023)
- [Visible and infrared self-supervised fusion trained on a single example](https://mlsanity.com/paper/2307.04100/) (2023)
- [Multi-Sensor Matching with HyperNetworks](https://mlsanity.com/paper/2601.12325/) (2026)
- [CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space](https://mlsanity.com/paper/2609.00272/) (2026)

## Abstract (arXiv)

In this work, we propose a novel Convolutional Neural Network (CNN) architecture for the joint detection and matching of feature points in images acquired by different sensors using a single forward pass. The resulting feature detector is tightly coupled with the feature descriptor, in contrast to classical approaches (SIFT, etc.), where the detection phase precedes and differs from computing the descriptor. Our approach utilizes two CNN subnetworks, the first being a Siamese CNN and the second, consisting of dual non-weight-sharing CNNs. This allows simultaneous processing and fusion of the joint and disjoint cues in the multimodal image patches. The proposed approach is experimentally shown to outperform contemporary state-of-the-art schemes when applied to multiple datasets of multimodal images. It is also shown to provide repeatable feature points detections across multisensor images, outperforming state-of-the-art detectors. To the best of our knowledge, it is the first unified approach for the detection and matching of such images.