# PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration

arXiv: https://arxiv.org/abs/2106.05124 · HTML: https://mlsanity.com/paper/2106.05124/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 97 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection](https://mlsanity.com/paper/1802.09972/) (2018)
- [Gabor Convolutional Networks](https://mlsanity.com/paper/1705.01450/) (2017)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity](https://mlsanity.com/paper/2103.16871/) (2021)
- [DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation](https://mlsanity.com/paper/1604.07944/) (2016)
- [Normalized Total Gradient: A New Measure for Multispectral Image Registration](https://mlsanity.com/paper/1702.04562/) (2017)

## Built on by (most cited first, 6 of 6)

- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [HMPE:HeatMap Embedding for Efficient Transformer-Based Small Object Detection](https://mlsanity.com/paper/2504.13469/) (2025)
- [Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection](https://mlsanity.com/paper/2604.15065/) (2026)
- [Relational Representation Learning Network for Cross-Spectral Image Patch Matching](https://mlsanity.com/paper/2403.11751/) (2024)
- [Reliable Projection Based Unsupervised Learning for Semi-Definite QCQP with Application of Beamforming Optimization](https://mlsanity.com/paper/2407.03668/) (2024)
- [Multi-Expert Learning Framework with the State Space Model for Optical and SAR Image Registration](https://mlsanity.com/paper/2507.04397/) (2025)

## Abstract (arXiv)

Multispectral and multimodal images are of important usage in the field of multi-source visual information fusion. Due to the alternation or movement of image devices, the acquired multispectral and multimodal images are usually misaligned, and hence image registration is pre-requisite. Different from the registration of common images, the registration of multispectral or multimodal images is a challenging problem due to the nonlinear variation of intensity and gradient. To cope with this challenge, we propose the phase congruency network (PCNet) to enhance the structure similarity of multispectral or multimodal images. The images can then be aligned using the similarity-enhanced feature maps produced by the network. PCNet is constructed under the inspiration of the well-known phase congruency. The network embeds the phase congruency prior into two simple trainable layers and series of modified learnable Gabor kernels. Thanks to the prior knowledge, once trained, PCNet is applicable on a variety of multispectral and multimodal data such as flash/no-flash and RGB/NIR images without additional further tuning. The prior also makes the network lightweight. The trainable parameters of PCNet are 2400 times less than the deep-learning registration method DHN, while its registration performance surpasses DHN. Experimental results validate that PCNet outperforms current state-of-the-art conventional multimodal registration algorithms. Besides, PCNet can act as a complementary part of the deep-learning registration methods, which significantly boosts their registration accuracy. The percentage of the number of images under 1 pixel average corner error (ACE) of UDHN is raised from 0.2% to 89.9% after the processing of PCNet.