# SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning

arXiv: https://arxiv.org/abs/2407.08148 · HTML: https://mlsanity.com/paper/2407.08148/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Omnivore: A Single Model for Many Visual Modalities](https://mlsanity.com/paper/2201.08377/) (2022)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [UniT: Multimodal Multitask Learning with a Unified Transformer](https://mlsanity.com/paper/2102.10772/) (2021)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Iterative Deep Homography Estimation](https://mlsanity.com/paper/2203.15982/) (2022)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection](https://mlsanity.com/paper/2103.15346/) (2021)
- [LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation](https://mlsanity.com/paper/2106.04067/) (2021)
- [Unsupervised Homography Estimation with Coplanarity-Aware GAN](https://mlsanity.com/paper/2205.03821/) (2022)
- [Perceptual Loss for Robust Unsupervised Homography Estimation](https://mlsanity.com/paper/2104.10011/) (2021)
- [Normalized Total Gradient: A New Measure for Multispectral Image Registration](https://mlsanity.com/paper/1702.04562/) (2017)
- [Multimodal-Boost: Multimodal Medical Image Super-Resolution using Multi-Attention Network with Wavelet Transform](https://mlsanity.com/paper/2110.11684/) (2021)

## Built on by (most cited first, 7 of 7)

- [SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization](https://mlsanity.com/paper/2409.17993/) (2024)
- [Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint](https://mlsanity.com/paper/2509.24423/) (2025)
- [Towards Generalized Multimodal Homography Estimation](https://mlsanity.com/paper/2603.03956/) (2026)
- [HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps](https://mlsanity.com/paper/2601.02730/) (2026)
- [Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations](https://mlsanity.com/paper/2602.18822/) (2026)
- [Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement](https://mlsanity.com/paper/2603.19623/) (2026)
- [CamFlow+: Hybrid Motion Bases for 2D Camera Motion Estimation with Stabilization Applications](https://mlsanity.com/paper/2606.05915/) (2026)

## Abstract (arXiv)

We propose a novel unsupervised cross-modal homography estimation framework based on intra-modal Self-supervised learning, Correlation, and consistent feature map Projection, namely SCPNet. The concept of intra-modal self-supervised learning is first presented to facilitate the unsupervised cross-modal homography estimation. The correlation-based homography estimation network and the consistent feature map projection are combined to form the learnable architecture of SCPNet, boosting the unsupervised learning framework. SCPNet is the first to achieve effective unsupervised homography estimation on the satellite-map image pair cross-modal dataset, GoogleMap, under [-32,+32] offset on a 128x128 image, leading the supervised approach MHN by 14.0% of mean average corner error (MACE). We further conduct extensive experiments on several cross-modal/spectral and manually-made inconsistent datasets, on which SCPNet achieves the state-of-the-art (SOTA) performance among unsupervised approaches, and owns 49.0%, 25.2%, 36.4%, and 10.7% lower MACEs than the supervised approach MHN. Source code is available at https://github.com/RM-Zhang/SCPNet.