# SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization

arXiv: https://arxiv.org/abs/2409.17993 · HTML: https://mlsanity.com/paper/2409.17993/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models](https://mlsanity.com/paper/2205.07680/) (2022)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.05358/) (2021)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator](https://mlsanity.com/paper/2308.16906/) (2023)
- [Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images](https://mlsanity.com/paper/2106.12859/) (2021)
- [Iterative Deep Homography Estimation](https://mlsanity.com/paper/2203.15982/) (2022)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection](https://mlsanity.com/paper/2103.15346/) (2021)
- [LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation](https://mlsanity.com/paper/2106.04067/) (2021)
- [SSDA-YOLO: Semi-supervised Domain Adaptive YOLO for Cross-Domain Object Detection](https://mlsanity.com/paper/2211.02213/) (2022)
- [Perceptual Loss for Robust Unsupervised Homography Estimation](https://mlsanity.com/paper/2104.10011/) (2021)
- [A Deep Primal-Dual Network for Guided Depth Super-Resolution](https://mlsanity.com/paper/1607.08569/) (2016)
- [SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning](https://mlsanity.com/paper/2407.08148/) (2024)
- [Multimodal-Boost: Multimodal Medical Image Super-Resolution using Multi-Attention Network with Wavelet Transform](https://mlsanity.com/paper/2110.11684/) (2021)

## Built on by (most cited first, 4 of 4)

- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint](https://mlsanity.com/paper/2509.24423/) (2025)
- [Towards Generalized Multimodal Homography Estimation](https://mlsanity.com/paper/2603.03956/) (2026)
- [HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps](https://mlsanity.com/paper/2601.02730/) (2026)

## Abstract (arXiv)

We propose a novel unsupervised cross-modal homography estimation learning framework, named Split Supervised Homography estimation Network (SSHNet). SSHNet reformulates the unsupervised cross-modal homography estimation into two supervised sub-problems, each addressed by its specialized network: a homography estimation network and a modality transfer network. To realize stable training, we introduce an effective split optimization strategy to train each network separately within its respective sub-problem. We also formulate an extra homography feature space supervision to enhance feature consistency, further boosting the estimation accuracy. Moreover, we employ a simple yet effective distillation training technique to reduce model parameters and improve cross-domain generalization ability while maintaining comparable performance. The training stability of SSHNet enables its cooperation with various homography estimation architectures. Experiments reveal that the SSHNet using IHN as homography estimation network, namely SSHNet-IHN, outperforms previous unsupervised approaches by a significant margin. Even compared to supervised approaches MHN and LocalTrans, SSHNet-IHN achieves 47.4% and 85.8% mean average corner errors (MACEs) reduction on the challenging OPT-SAR dataset.