mlsanityData to 7 Oct 2026

Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network

arXiv 2606.03341 ↗First posted 2 Jun 2026cs.CVcited by 0

On the map: State-space sequence models (S4, Mamba) → Point clouds, detection and remote sensing

Abstract (arXiv)

In multi-modal image registration, the primary challenge lies in shared structural information extraction. Compared to Transformers, Structured State Space Duality (SSD) offers greater global structural feature extraction with higher efficiency during training and inference. Inspired by these advantages, we propose a novel algorithm for multi-modal image registration, named RegNetMamba-2. Our algorithm incorporates SSD into coarse-to-fine matching process to extract local and global structural features effectively. Firstly, SSD is applied in three different scales for multi-modal feature extraction in our network. To strengthen local representation, we pay more attention on foreground edge and structural information by feature scaling function of SSD. Secondly, for shared feature extraction of input images and multi-modal feature fusion in all scales, we propose cross-modality feature fusion model based on SSD, consisting of Cross-Modality feature Interaction (CMI) module and Multi-Scale feature Fusion (MSF) module. CMI module is designed for cross-modality feature extraction of each scale by SSD in cross form. MSF module is designed to employ a progressive upward fusion in feature-level to obtain fine features, consisting of multi-modal features in all scales. Following coarse-to-fine, the features in 1/8 scale from CMI and 1/2 scale from MSF are collected to calculate matching probability scores. Then we respectively establish matching process by correspondences of pixel-wise. Extensive experiments demonstrate that comparing with state-of-the-art deep-learning based algorithms, RegNetMamba-2 has achieved good effects in both performance and efficiency for multi-modal image registration on the following datasets: VIS-SAR (OSDataset), VIS-IR (LGHD/RoadSence) and VIS-NIR (RGB-NIR sense).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 27 of its 48 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

27 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024COMO: Cross-Mamba Interaction and Offset-Guided Fusion for Multimodal Object Detection24 Dec

Other references · 26

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  3. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  6. 2024VMamba: Visual State Space Modelcited by 955
  7. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  8. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 324
  9. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 235
  10. 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 170
  11. 2023FLatten Transformer: Vision Transformer using Focused Linear Attentioncited by 170
  12. 2022MatchFormer: Interleaving Attention in Transformers for Feature Matchingcited by 126
14 more
  1. 2018MAGSAC: marginalizing sample consensuscited by 122
  2. 2024Demystify Mamba in Vision: A Linear Attention Perspectivecited by 101
  3. 2021Learning to Match Features with Seeded Graph Matching Networkcited by 92
  4. 2022ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matchingcited by 62
  5. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Modelcited by 62
  6. 2024OmniGlue: Generalizable Feature Matching with Foundation Model Guidancecited by 55
  7. 2024XoFTR: Cross-modal Feature Matching Transformercited by 45
  8. 2024VSSD: Vision Mamba with Non-Causal State Space Dualitycited by 29
  9. 2022ReDFeat: Recoupling Detection and Description for Multimodal Feature Learningcited by 26
  10. 2023Learning Rotation-Equivariant Features for Visual Correspondencecited by 23
  11. 2023Adaptive Spot-Guided Transformer for Consistent Local Feature Matchingcited by 22
  12. 2025JamMa: Ultra-lightweight Local Feature Matching with Joint Mambacited by 14
  13. 2022FeatureBooster: Boosting Feature Descriptors with a Lightweight Neural Networkcited by 10
  14. 2024LoFLAT: Local Feature Matching using Focused Linear Attention Transformercited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024VMambaMorph: a Multi-Modality Deformable Image Registration Framework based on Visual State Space Model with Cross-Scan Module
  2. 2025MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching
  3. 2026CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration
  4. 2025DM3T: Harmonizing Modalities via Diffusion for Multi-Object Tracking
  5. 2025Modality Translation and Registration of MR and Ultrasound Images Using Diffusion Models
  6. 2024AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment
  7. 2025Multi-Expert Learning Framework with the State Space Model for Optical and SAR Image Registration
  8. 2025Collaborative Learning for Unsupervised Multimodal Remote Sensing Image Registration: Integrating Self-Supervision and MIM-Guided Diffusion-Based Image Translation