# Handheld Multi-Frame Super-Resolution

arXiv: https://arxiv.org/abs/1905.03277 · HTML: https://mlsanity.com/paper/1905.03277/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2019.

## Built on by (most cited first, 30 of 85)

- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [Mobile Computational Photography: A Tour](https://mlsanity.com/paper/2102.09000/) (2021)
- [Model-Based Image Signal Processors via Learnable Dictionaries](https://mlsanity.com/paper/2201.03210/) (2022)
- [Deep Reparametrization of Multi-Frame Super-Resolution and Denoising](https://mlsanity.com/paper/2108.08286/) (2021)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [Burstormer: Burst Image Restoration and Enhancement Transformer](https://mlsanity.com/paper/2304.01194/) (2023)
- [SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation](https://mlsanity.com/paper/2104.10325/) (2021)
- [Revisiting Temporal Alignment for Video Restoration](https://mlsanity.com/paper/2111.15288/) (2021)
- [NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results](https://mlsanity.com/paper/2106.03839/) (2021)
- [Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2104.06191/) (2021)
- [Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution](https://mlsanity.com/paper/2212.05342/) (2022)
- [Multi-Spectral Multi-Image Super-Resolution of Sentinel-2 with Radiometric Consistency Losses and Its Effect on Building Delineation](https://mlsanity.com/paper/2111.03231/) (2021)
- [Neural Image Representations for Multi-Image Fusion and Layer Separation](https://mlsanity.com/paper/2108.01199/) (2021)
- [High Dynamic Range and Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2207.14671/) (2022)
- [Adaptive Patch Exiting for Scalable Single Image Super-Resolution](https://mlsanity.com/paper/2203.11589/) (2022)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [Toward Real-World Super-Resolution via Adaptive Downsampling Models](https://mlsanity.com/paper/2109.03444/) (2021)
- [NTIRE 2025 Challenge on Efficient Burst HDR and Restoration: Datasets, Methods, and Results](https://mlsanity.com/paper/2505.12089/) (2025)
- [The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement](https://mlsanity.com/paper/2111.13738/) (2021)
- [DC2: Dual-Camera Defocus Control by Learning to Refocus](https://mlsanity.com/paper/2304.03285/) (2023)
- [Towards Real-World Burst Image Super-Resolution: Benchmark and Method](https://mlsanity.com/paper/2309.04803/) (2023)
- [Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs](https://mlsanity.com/paper/2404.10700/) (2024)
- [A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift](https://mlsanity.com/paper/2203.09294/) (2022)
- [Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites](https://mlsanity.com/paper/2205.02031/) (2022)
- [Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography](https://mlsanity.com/paper/2212.12324/) (2022)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-Trained Siamese Transformers](https://mlsanity.com/paper/2112.09426/) (2021)

## Abstract (arXiv)

Compared to DSLR cameras, smartphone cameras have smaller sensors, which limits their spatial resolution; smaller apertures, which limits their light gathering ability; and smaller pixels, which reduces their signal-to noise ratio. The use of color filter arrays (CFAs) requires demosaicing, which further degrades resolution. In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multiframe super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images. We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio. Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones. Specifically, the algorithm is the basis of the Super-Res Zoom feature, as well as the default merge method in Night Sight mode (whether zooming or not) on Google's flagship phone.