# LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation

arXiv: https://arxiv.org/abs/2106.04067 · HTML: https://mlsanity.com/paper/2106.04067/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [MUTAN: Multimodal Tucker Fusion for Visual Question Answering](https://mlsanity.com/paper/1705.06676/) (2017)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [Image Super-Resolution by Neural Texture Transfer](https://mlsanity.com/paper/1903.00834/) (2019)
- [Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering](https://mlsanity.com/paper/1708.01471/) (2017)
- [CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping](https://mlsanity.com/paper/1807.10547/) (2018)
- [Multimodal Transformer with Multi-View Visual Representation for Image Captioning](https://mlsanity.com/paper/1905.07841/) (2019)
- [ContextDesc: Local Descriptor Augmentation with Cross-Modality Context](https://mlsanity.com/paper/1904.04084/) (2019)
- [Attention-Based Multimodal Fusion for Video Description](https://mlsanity.com/paper/1701.03126/) (2017)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Repeatability Is Not Enough: Learning Affine Regions via Discriminability](https://mlsanity.com/paper/1711.06704/) (2017)
- [GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints](https://mlsanity.com/paper/1807.06294/) (2018)

## Built on by (most cited first, 28 of 28)

- [Iterative Deep Homography Estimation](https://mlsanity.com/paper/2203.15982/) (2022)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Unsupervised Homography Estimation with Coplanarity-Aware GAN](https://mlsanity.com/paper/2205.03821/) (2022)
- [Semi-supervised Deep Large-baseline Homography Estimation with Progressive Equivalence Constraint](https://mlsanity.com/paper/2212.02763/) (2022)
- [GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning](https://mlsanity.com/paper/2301.10018/) (2023)
- [Cross-Modal Learning for Anomaly Detection in Complex Industrial Process: Methodology and Benchmark](https://mlsanity.com/paper/2406.09016/) (2024)
- [STHN: Deep Homography Estimation for UAV Thermal Geo-localization with Satellite Imagery](https://mlsanity.com/paper/2405.20470/) (2024)
- [SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning](https://mlsanity.com/paper/2407.08148/) (2024)
- [Supervised Homography Learning with Realistic Dataset Generation](https://mlsanity.com/paper/2307.15353/) (2023)
- [PRISE: Demystifying Deep Lucas-Kanade with Strongly Star-Convex Constraints for Multimodel Image Alignment](https://mlsanity.com/paper/2303.11526/) (2023)
- [SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization](https://mlsanity.com/paper/2409.17993/) (2024)
- [UASTHN: Uncertainty-Aware Deep Homography Estimation for UAV Satellite-Thermal Geo-localization](https://mlsanity.com/paper/2502.01035/) (2025)
- [Fast and Interpretable 2D Homography Decomposition: Similarity-Kernel-Similarity and Affine-Core-Affine Transformations](https://mlsanity.com/paper/2402.18008/) (2024)
- [Deep Loss Convexification for Learning Iterative Models](https://mlsanity.com/paper/2411.10649/) (2024)
- [RePoseDM: Recurrent Pose Alignment and Gradient Guidance for Pose Guided Image Synthesis](https://mlsanity.com/paper/2310.16074/) (2023)
- [CodingHomo: Bootstrapping Deep Homography With Video Coding](https://mlsanity.com/paper/2504.12165/) (2025)
- [SSORN: Self-Supervised Outlier Removal Network for Robust Homography Estimation](https://mlsanity.com/paper/2208.14093/) (2022)
- [Parallax-Tolerant Image Stitching with Epipolar Displacement Field](https://mlsanity.com/paper/2311.16637/) (2023)
- [MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration](https://mlsanity.com/paper/2411.01399/) (2024)
- [SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement](https://mlsanity.com/paper/2412.06352/) (2024)
- [Auto-regressive transformation for image alignment](https://mlsanity.com/paper/2505.04864/) (2025)
- [HomoFM: Deep Homography Estimation with Flow Matching](https://mlsanity.com/paper/2601.18222/) (2026)
- [Decoupled Geometric Parameterization and its Application in Deep Homography Estimation](https://mlsanity.com/paper/2505.16599/) (2025)
- [Estimating 2D Camera Motion with Hybrid Motion Basis](https://mlsanity.com/paper/2507.22480/) (2025)
- [ShelfRectNet: Single View Shelf Image Rectification with Homography Estimation](https://mlsanity.com/paper/2511.20335/) (2025)
- [CamFlow+: Hybrid Motion Bases for 2D Camera Motion Estimation with Stabilization Applications](https://mlsanity.com/paper/2606.05915/) (2026)
- [SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios](https://mlsanity.com/paper/2606.30408/) (2026)

## Abstract (arXiv)

Cross-resolution image alignment is a key problem in multiscale gigapixel photography, which requires to estimate homography matrix using images with large resolution gap. Existing deep homography methods concatenate the input images or features, neglecting the explicit formulation of correspondences between them, which leads to degraded accuracy in cross-resolution challenges. In this paper, we consider the cross-resolution homography estimation as a multimodal problem, and propose a local transformer network embedded within a multiscale structure to explicitly learn correspondences between the multimodal inputs, namely, input images with different resolutions. The proposed local transformer adopts a local attention map specifically for each position in the feature. By combining the local transformer with the multiscale structure, the network is able to capture long-short range correspondences efficiently and accurately. Experiments on both the MS-COCO dataset and the real-captured cross-resolution dataset show that the proposed network outperforms existing state-of-the-art feature-based and deep-learning-based homography estimation methods, and is able to accurately align images under 10× resolution gap.