# Deep Reparametrization of Multi-Frame Super-Resolution and Denoising

arXiv: https://arxiv.org/abs/2108.08286 · HTML: https://mlsanity.com/paper/2108.08286/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 69 references that are in the map.
- First posted on arXiv 18 Aug 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Replacing Mobile Camera ISP with a Single Deep Learning Model](https://mlsanity.com/paper/2002.05509/) (2020)
- [Regularization by Denoising: Clarifications and New Interpretations](https://mlsanity.com/paper/1806.02296/) (2018)
- [Learning What to Learn for Video Object Segmentation](https://mlsanity.com/paper/2003.11540/) (2020)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Handheld Mobile Photography in Very Low Light](https://mlsanity.com/paper/1910.11336/) (2019)
- [Deep Burst Denoising](https://mlsanity.com/paper/1712.05790/) (2017)
- [Basis Prediction Networks for Effective Burst Denoising with Large Kernels](https://mlsanity.com/paper/1912.04421/) (2019)
- [HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery](https://mlsanity.com/paper/2002.06460/) (2020)
- [Deep Image Demosaicking using a Cascade of Convolutional Residual Denoising Networks](https://mlsanity.com/paper/1803.05215/) (2018)
- [DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images](https://mlsanity.com/paper/1907.06490/) (2019)
- [Iterative Residual CNNs for Burst Photography Applications](https://mlsanity.com/paper/1811.12197/) (2018)

## Built on by (most cited first, 30 of 39)

- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [NAN: Noise-Aware NeRFs for Burst-Denoising](https://mlsanity.com/paper/2204.04668/) (2022)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [Burstormer: Burst Image Restoration and Enhancement Transformer](https://mlsanity.com/paper/2304.01194/) (2023)
- [QMambaBSR: Burst Image Super-Resolution with Query State Space Model](https://mlsanity.com/paper/2408.08665/) (2024)
- [High Dynamic Range and Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2207.14671/) (2022)
- [Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments](https://mlsanity.com/paper/2210.16046/) (2022)
- [Towards Real-World Burst Image Super-Resolution: Benchmark and Method](https://mlsanity.com/paper/2309.04803/) (2023)
- [Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement](https://mlsanity.com/paper/2304.06703/) (2023)
- [BurstM: Deep Burst Multi-scale SR using Fourier Space with Optical Flow](https://mlsanity.com/paper/2409.15384/) (2024)
- [Dual-Camera Joint Deblurring-Denoising](https://mlsanity.com/paper/2309.08826/) (2023)
- [Specularity Factorization for Low-Light Enhancement](https://mlsanity.com/paper/2404.01998/) (2024)
- [CRNet: A Detail-Preserving Network for Unified Image Restoration and Enhancement Task](https://mlsanity.com/paper/2404.14132/) (2024)
- [Kernel-aware Burst Blind Super-Resolution](https://mlsanity.com/paper/2112.07315/) (2021)
- [Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery](https://mlsanity.com/paper/2303.05879/) (2023)
- [Efficient View Synthesis and 3D-based Multi-Frame Denoising with Multiplane Feature Representations](https://mlsanity.com/paper/2303.18139/) (2023)
- [RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution](https://mlsanity.com/paper/2306.17595/) (2023)
- [PRISE: Demystifying Deep Lucas-Kanade with Strongly Star-Convex Constraints for Multimodel Image Alignment](https://mlsanity.com/paper/2303.11526/) (2023)
- [Deep Loss Convexification for Learning Iterative Models](https://mlsanity.com/paper/2411.10649/) (2024)
- [Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis](https://mlsanity.com/paper/2503.18705/) (2025)
- [Improving Bracket Image Restoration and Enhancement with Flow-guided Alignment and Enhanced Feature Aggregation](https://mlsanity.com/paper/2404.10358/) (2024)
- [A Modular Conditional Diffusion Framework for Image Reconstruction](https://mlsanity.com/paper/2411.05993/) (2024)
- [One-Pot Multi-Frame Denoising](https://mlsanity.com/paper/2302.11544/) (2023)
- [Burst Image Super-Resolution with Base Frame Selection](https://mlsanity.com/paper/2406.17869/) (2024)
- [Towards Degradation-Robust Reconstruction in Generalizable NeRF](https://mlsanity.com/paper/2411.11691/) (2024)
- [MFSR-GAN: Multi-Frame Super-Resolution with Handheld Motion Modeling](https://mlsanity.com/paper/2502.20824/) (2025)
- [DenoiseGS: Gaussian Reconstruction Model for Burst Denoising](https://mlsanity.com/paper/2511.22939/) (2025)
- [SuperF: Neural Implicit Fields for Multi-Image Super-Resolution](https://mlsanity.com/paper/2512.09115/) (2025)
- [DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis](https://mlsanity.com/paper/2602.23022/) (2026)
- [Uncertainty-Aware Cascaded Dilation Filtering for High-Efficiency Deraining](https://mlsanity.com/paper/2201.02366/) (2022)

## Abstract (arXiv)

We propose a deep reparametrization of the maximum a posteriori formulation commonly employed in multi-frame image restoration tasks. Our approach is derived by introducing a learned error metric and a latent representation of the target image, which transforms the MAP objective to a deep feature space. The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction. Our approach thereby leverages the advantages of deep learning, while also benefiting from the principled multi-frame fusion provided by the classical MAP formulation. We validate our approach through comprehensive experiments on burst denoising and burst super-resolution datasets. Our approach sets a new state-of-the-art for both tasks, demonstrating the generality and effectiveness of the proposed formulation.