# DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images

arXiv: https://arxiv.org/abs/1907.06490 · HTML: https://mlsanity.com/paper/1907.06490/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 30 of 30)

- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [Deep Reparametrization of Multi-Frame Super-Resolution and Denoising](https://mlsanity.com/paper/2108.08286/) (2021)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2104.06191/) (2021)
- [Multi-Spectral Multi-Image Super-Resolution of Sentinel-2 with Radiometric Consistency Losses and Its Effect on Building Delineation](https://mlsanity.com/paper/2111.03231/) (2021)
- [Permutation invariance and uncertainty in multitemporal image super-resolution](https://mlsanity.com/paper/2105.12409/) (2021)
- [MuS2: A Real-World Benchmark for Sentinel-2 Multi-Image Super-Resolution](https://mlsanity.com/paper/2210.02745/) (2022)
- [Multitemporal and multispectral data fusion for super-resolution of Sentinel-2 images](https://mlsanity.com/paper/2301.11154/) (2023)
- [Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites](https://mlsanity.com/paper/2205.02031/) (2022)
- [Kernel-aware Burst Blind Super-Resolution](https://mlsanity.com/paper/2112.07315/) (2021)
- [Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing](https://mlsanity.com/paper/2311.18082/) (2023)
- [Burst Super-Resolution with Diffusion Models for Improving Perceptual Quality](https://mlsanity.com/paper/2403.19428/) (2024)
- [Proba-V-ref: Repurposing the Proba-V challenge for reference-aware super resolution](https://mlsanity.com/paper/2101.10200/) (2021)
- [A Feature Reuse Framework with Texture-adaptive Aggregation for Reference-based Super-Resolution](https://mlsanity.com/paper/2306.01500/) (2023)
- [Wavelet-Assisted Mamba for Satellite-Derived Sea Surface Temperature Super-Resolution](https://mlsanity.com/paper/2509.24334/) (2025)
- [Multi-temporal speckle reduction with self-supervised deep neural networks](https://mlsanity.com/paper/2207.11095/) (2022)
- [Deep 3D World Models for Multi-Image Super-Resolution Beyond Optical Flow](https://mlsanity.com/paper/2401.16972/) (2024)
- [Super-Resolving Beyond Satellite Hardware Using Realistically Degraded Images](https://mlsanity.com/paper/2103.06270/) (2021)
- [Multi-image Super-resolution via Quality Map Associated Attention Network](https://mlsanity.com/paper/2202.13124/) (2022)
- [Super-resolved multi-temporal segmentation with deep permutation-invariant networks](https://mlsanity.com/paper/2204.02631/) (2022)
- [Combination of Single and Multi-frame Image Super-resolution: An Analytical Perspective](https://mlsanity.com/paper/2303.03212/) (2023)
- [CoT-MISR:Marrying Convolution and Transformer for Multi-Image Super-Resolution](https://mlsanity.com/paper/2303.06548/) (2023)
- [CANDID: Correspondence AligNment for Deep-burst Image Denoising](https://mlsanity.com/paper/2306.09887/) (2023)
- [ESC-MISR: Enhancing Spatial Correlations for Multi-Image Super-Resolution in Remote Sensing](https://mlsanity.com/paper/2411.04706/) (2024)
- [A Single-Frame and Multi-Frame Cascaded Image Super-Resolution Method](https://mlsanity.com/paper/2412.09846/) (2024)
- [Feature Alignment with Equivariant Convolutions for Burst Image Super-Resolution](https://mlsanity.com/paper/2503.08300/) (2025)
- [Burst Image Super-Resolution via Multi-Cross Attention Encoding and Multi-Scan State-Space Decoding](https://mlsanity.com/paper/2505.19668/) (2025)
- [SatFusion: A Unified Framework for Enhancing Remote Sensing Images via Multi-Frame and Multi-Source Images Fusion](https://mlsanity.com/paper/2510.07905/) (2025)
- [Dual-Branch State-Displacement Network for Sea Surface Temperature Super-Resolution](https://mlsanity.com/paper/2608.15423/) (2026)

## Abstract (arXiv)

Recently, convolutional neural networks (CNN) have been successfully applied to many remote sensing problems. However, deep learning techniques for multi-image super-resolution from multitemporal unregistered imagery have received little attention so far. This work proposes a novel CNN-based technique that exploits both spatial and temporal correlations to combine multiple images. This novel framework integrates the spatial registration task directly inside the CNN, and allows to exploit the representation learning capabilities of the network to enhance registration accuracy. The entire super-resolution process relies on a single CNN with three main stages: shared 2D convolutions to extract high-dimensional features from the input images; a subnetwork proposing registration filters derived from the high-dimensional feature representations; 3D convolutions for slow fusion of the features from multiple images. The whole network can be trained end-to-end to recover a single high resolution image from multiple unregistered low resolution images. The method presented in this paper is the winner of the PROBA-V super-resolution challenge issued by the European Space Agency.