# XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration

arXiv: https://arxiv.org/abs/2411.07430 · HTML: https://mlsanity.com/paper/2411.07430/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [RoMa: Robust Dense Feature Matching](https://mlsanity.com/paper/2305.15404/) (2023)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [XFeat: Accelerated Features for Lightweight Image Matching](https://mlsanity.com/paper/2404.19174/) (2024)
- [GlueStick: Robust Image Matching by Sticking Points and Lines Together](https://mlsanity.com/paper/2304.02008/) (2023)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Iterative Deep Homography Estimation](https://mlsanity.com/paper/2203.15982/) (2022)
- [PMatch: Paired Masked Image Modeling for Dense Geometric Matching](https://mlsanity.com/paper/2303.17342/) (2023)
- [ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning](https://mlsanity.com/paper/2205.07439/) (2022)
- [RIFT2: Speeding-up RIFT with A New Rotation-Invariance Technique](https://mlsanity.com/paper/2303.00319/) (2023)
- [Joint detection and matching of feature points in multimodal images](https://mlsanity.com/paper/1810.12941/) (2018)
- [AbHE: All Attention-based Homography Estimation](https://mlsanity.com/paper/2212.03029/) (2022)
- [VisIRNet: Deep Image Alignment for UAV-taken Visible and Infrared Image Pairs](https://mlsanity.com/paper/2402.09635/) (2024)

## Built on by (most cited first, 2 of 2)

- [DistillMatch: Leveraging Knowledge Distillation from Vision Foundation Model for Multimodal Image Matching](https://mlsanity.com/paper/2509.16017/) (2025)
- [Learning Cross-Spectral Point Features with Task-Oriented Training](https://mlsanity.com/paper/2505.12593/) (2025)

## Abstract (arXiv)

Accurate multispectral image matching presents significant challenges due to non-linear intensity variations across spectral modalities, extreme viewpoint changes, and the scarcity of labeled datasets. Current state-of-the-art methods are typically specialized for a single spectral difference, such as visibleinfrared, and struggle to adapt to other modalities due to their reliance on expensive supervision, such as depth maps or camera poses. To address the need for rapid adaptation across modalities, we introduce XPoint, a self-supervised, modular image-matching framework designed for adaptive training and fine-tuning on aligned multispectral datasets, allowing users to customize key components based on their specific tasks. XPoint employs modularity and self-supervision to allow for the adjustment of elements such as the base detector, which generates pseudoground truth keypoints invariant to viewpoint and spectrum variations. The framework integrates a VMamba encoder, pretrained on segmentation tasks, for robust feature extraction, and includes three joint decoder heads: two are dedicated to interest point and descriptor extraction; and a task-specific homography regression head imposes geometric constraints for superior performance in tasks like image registration. This flexible architecture enables quick adaptation to a wide range of modalities, demonstrated by training on Optical-Thermal data and fine-tuning on settings such as visual-near infrared, visual-infrared, visual-longwave infrared, and visual-synthetic aperture radar. Experimental results show that XPoint consistently outperforms or matches state-ofthe-art methods in feature matching and image registration tasks across five distinct multispectral datasets. Our source code is available at https://github.com/canyagmur/XPoint.