# VisIRNet: Deep Image Alignment for UAV-taken Visible and Infrared Image Pairs

arXiv: https://arxiv.org/abs/2402.09635 · HTML: https://mlsanity.com/paper/2402.09635/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 61 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 7 of 7)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Relative Pose from SIFT Features](https://mlsanity.com/paper/2203.07930/) (2022)
- [Offloading Deep Learning Powered Vision Tasks from UAV to 5G Edge Server with Denoising](https://mlsanity.com/paper/2302.01991/) (2023)

## Built on by (most cited first, 2 of 2)

- [XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration](https://mlsanity.com/paper/2411.07430/) (2024)
- [Survey on Disaster Management Datasets for Remote Sensing Based Emergency Applications](https://mlsanity.com/paper/2605.08196/) (2026)

## Abstract (arXiv)

This paper proposes a deep learning based solution for multi-modal image alignment regarding UAV-taken images. Many recently proposed state-of-the-art alignment techniques rely on using Lucas-Kanade (LK) based solutions for a successful alignment. However, we show that we can achieve state of the art results without using LK-based methods. Our approach carefully utilizes a two-branch based convolutional neural network (CNN) based on feature embedding blocks. We propose two variants of our approach, where in the first variant (ModelA), we directly predict the new coordinates of only the four corners of the image to be aligned; and in the second one (ModelB), we predict the homography matrix directly. Applying alignment on the image corners forces algorithm to match only those four corners as opposed to computing and matching many (key)points, since the latter may cause many outliers, yielding less accurate alignment. We test our proposed approach on four aerial datasets and obtain state of the art results, when compared to the existing recent deep LK-based architectures.