# Permutation invariance and uncertainty in multitemporal image super-resolution

arXiv: https://arxiv.org/abs/2105.12409 · HTML: https://mlsanity.com/paper/2105.12409/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deeply-Recursive Convolutional Network for Image Super-Resolution](https://mlsanity.com/paper/1511.04491/) (2015)
- [Blind Super-Resolution Kernel Estimation using an Internal-GAN](https://mlsanity.com/paper/1909.06581/) (2019)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images](https://mlsanity.com/paper/1907.06490/) (2019)
- [Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks](https://mlsanity.com/paper/2007.03107/) (2020)
- [Super-Resolution of PROBA-V Images Using Convolutional Neural Networks](https://mlsanity.com/paper/1907.01821/) (2019)
- [DeepSUM++: Non-local Deep Neural Network for Super-Resolution of Unregistered Multitemporal Images](https://mlsanity.com/paper/2001.06342/) (2020)

## Built on by (most cited first, 18 of 18)

- [Diffusion Models, Image Super-Resolution And Everything: A Survey](https://mlsanity.com/paper/2401.00736/) (2024)
- [Open High-Resolution Satellite Imagery: The WorldStrat Dataset -- With Application to Super-Resolution](https://mlsanity.com/paper/2207.06418/) (2022)
- [Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances](https://mlsanity.com/paper/2209.13131/) (2022)
- [High-Resolution Building and Road Detection from Sentinel-2](https://mlsanity.com/paper/2310.11622/) (2023)
- [MuS2: A Real-World Benchmark for Sentinel-2 Multi-Image Super-Resolution](https://mlsanity.com/paper/2210.02745/) (2022)
- [Multitemporal and multispectral data fusion for super-resolution of Sentinel-2 images](https://mlsanity.com/paper/2301.11154/) (2023)
- [Graph Neural Networks Extract High-Resolution Cultivated Land Maps from Sentinel-2 Image Series](https://mlsanity.com/paper/2208.02349/) (2022)
- [A Study in Dataset Pruning for Image Super-Resolution](https://mlsanity.com/paper/2403.17083/) (2024)
- [Onboard Hyperspectral Super-Resolution with Deep Pushbroom Neural Network](https://mlsanity.com/paper/2507.20765/) (2025)
- [Multi-image Super-resolution via Quality Map Associated Attention Network](https://mlsanity.com/paper/2202.13124/) (2022)
- [Super-resolved multi-temporal segmentation with deep permutation-invariant networks](https://mlsanity.com/paper/2204.02631/) (2022)
- [CoT-MISR:Marrying Convolution and Transformer for Multi-Image Super-Resolution](https://mlsanity.com/paper/2303.06548/) (2023)
- [CANDID: Correspondence AligNment for Deep-burst Image Denoising](https://mlsanity.com/paper/2306.09887/) (2023)
- [ESC-MISR: Enhancing Spatial Correlations for Multi-Image Super-Resolution in Remote Sensing](https://mlsanity.com/paper/2411.04706/) (2024)
- [Toward task-driven satellite image super-resolution](https://mlsanity.com/paper/2503.15474/) (2025)
- [Keyframe-Centric State-Space Modeling for Burst Image Super-Resolution](https://mlsanity.com/paper/2503.19634/) (2025)
- [Task-driven real-world super-resolution of document scans](https://mlsanity.com/paper/2506.06953/) (2025)
- [Self-Supervised Uncertainty Estimation For Super-Resolution of Satellite Images](https://mlsanity.com/paper/2603.14074/) (2026)

## Abstract (arXiv)

Recent advances have shown how deep neural networks can be extremely effective at super-resolving remote sensing imagery, starting from a multitemporal collection of low-resolution images. However, existing models have neglected the issue of temporal permutation, whereby the temporal ordering of the input images does not carry any relevant information for the super-resolution task and causes such models to be inefficient with the, often scarce, ground truth data that available for training. Thus, models ought not to learn feature extractors that rely on temporal ordering. In this paper, we show how building a model that is fully invariant to temporal permutation significantly improves performance and data efficiency. Moreover, we study how to quantify the uncertainty of the super-resolved image so that the final user is informed on the local quality of the product. We show how uncertainty correlates with temporal variation in the series, and how quantifying it further improves model performance. Experiments on the Proba-V challenge dataset show significant improvements over the state of the art without the need for self-ensembling, as well as improved data efficiency, reaching the performance of the challenge winner with just 25% of the training data.