# Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts

arXiv: https://arxiv.org/abs/2104.06191 · HTML: https://mlsanity.com/paper/2104.06191/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 23 of 23)

- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Basis Prediction Networks for Effective Burst Denoising with Large Kernels](https://mlsanity.com/paper/1912.04421/) (2019)
- [HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery](https://mlsanity.com/paper/2002.06460/) (2020)
- [DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images](https://mlsanity.com/paper/1907.06490/) (2019)
- [Multi-Frame Super-Resolution Reconstruction with Applications to Medical Imaging](https://mlsanity.com/paper/1812.09375/) (2018)
- [Confidence-aware Levenberg-Marquardt optimization for joint motion estimation and super-resolution](https://mlsanity.com/paper/1609.01524/) (2016)

## Built on by (most cited first, 21 of 21)

- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [Burstormer: Burst Image Restoration and Enhancement Transformer](https://mlsanity.com/paper/2304.01194/) (2023)
- [QMambaBSR: Burst Image Super-Resolution with Query State Space Model](https://mlsanity.com/paper/2408.08665/) (2024)
- [High Dynamic Range and Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2207.14671/) (2022)
- [Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement](https://mlsanity.com/paper/2304.06703/) (2023)
- [Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites](https://mlsanity.com/paper/2205.02031/) (2022)
- [Dual-Camera Joint Deblurring-Denoising](https://mlsanity.com/paper/2309.08826/) (2023)
- [Kernel-aware Burst Blind Super-Resolution](https://mlsanity.com/paper/2112.07315/) (2021)
- [RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution](https://mlsanity.com/paper/2306.17595/) (2023)
- [Training Adaptive Reconstruction Networks for Blind Inverse Problems](https://mlsanity.com/paper/2202.11342/) (2022)
- [A Modular Conditional Diffusion Framework for Image Reconstruction](https://mlsanity.com/paper/2411.05993/) (2024)
- [Efficient RAW Image Deblurring with Adaptive Frequency Modulation](https://mlsanity.com/paper/2505.24407/) (2025)
- [Deep 3D World Models for Multi-Image Super-Resolution Beyond Optical Flow](https://mlsanity.com/paper/2401.16972/) (2024)
- [Burst Image Super-Resolution with Base Frame Selection](https://mlsanity.com/paper/2406.17869/) (2024)
- [DSR: Towards Drone Image Super-Resolution](https://mlsanity.com/paper/2208.12327/) (2022)
- [Towards Robust Drone Vision in the Wild](https://mlsanity.com/paper/2208.12655/) (2022)
- [Cross-Resolution Flow Propagation for Foveated Video Super-Resolution](https://mlsanity.com/paper/2212.13525/) (2022)
- [Fine Dense Alignment of Image Bursts through Camera Pose and Depth Estimation](https://mlsanity.com/paper/2312.05190/) (2023)
- [Enhanced Self-Supervised Multi-Image Super-Resolution for Camera Array Images](https://mlsanity.com/paper/2604.06816/) (2026)
- [MatRes: Zero-Shot Test-Time Model Adaptation for Simultaneous Matching and Restoration](https://mlsanity.com/paper/2604.10081/) (2026)

## Abstract (arXiv)

This presentation addresses the problem of reconstructing a high-resolution image from multiple lower-resolution snapshots captured from slightly different viewpoints in space and time. Key challenges for solving this problem include (i) aligning the input pictures with sub-pixel accuracy, (ii) handling raw (noisy) images for maximal faithfulness to native camera data, and (iii) designing/learning an image prior (regularizer) well suited to the task. We address these three challenges with a hybrid algorithm building on the insight from Wronski et al. that aliasing is an ally in this setting, with parameters that can be learned end to end, while retaining the interpretability of classical approaches to inverse problems. The effectiveness of our approach is demonstrated on synthetic and real image bursts, setting a new state of the art on several benchmarks and delivering excellent qualitative results on real raw bursts captured by smartphones and prosumer cameras.