# A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

arXiv: https://arxiv.org/abs/2203.09294 · HTML: https://mlsanity.com/paper/2203.09294/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [Neural Nearest Neighbors Networks](https://mlsanity.com/paper/1810.12575/) (2018)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes](https://mlsanity.com/paper/2003.14013/) (2020)
- [ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring](https://mlsanity.com/paper/2103.04260/) (2021)
- [Basis Prediction Networks for Effective Burst Denoising with Large Kernels](https://mlsanity.com/paper/1912.04421/) (2019)
- [Model-blind Video Denoising Via Frame-to-frame Training](https://mlsanity.com/paper/1811.12766/) (2018)
- [Iterative Joint Image Demosaicking and Denoising using a Residual Denoising Network](https://mlsanity.com/paper/1807.06403/) (2018)
- [Learning Deformable Kernels for Image and Video Denoising](https://mlsanity.com/paper/1904.06903/) (2019)
- [Joint Denoising and Demosaicking with Green Channel Prior for Real-world Burst Images](https://mlsanity.com/paper/2101.09870/) (2021)
- [A Review of an Old Dilemma: Demosaicking First, or Denoising First?](https://mlsanity.com/paper/2004.11577/) (2020)
- [Self-Supervised Poisson-Gaussian Denoising](https://mlsanity.com/paper/2002.09558/) (2020)
- [Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling](https://mlsanity.com/paper/2103.11744/) (2021)
- [Multi-Kernel Prediction Networks for Denoising of Burst Images](https://mlsanity.com/paper/1902.05392/) (2019)

## Built on by (most cited first, 10 of 10)

- [Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method](https://mlsanity.com/paper/2309.13890/) (2023)
- [D2HNet: Joint Denoising and Deblurring with Hierarchical Network for Robust Night Image Restoration](https://mlsanity.com/paper/2207.03294/) (2022)
- [NTIRE 2025 Challenge on Efficient Burst HDR and Restoration: Datasets, Methods, and Results](https://mlsanity.com/paper/2505.12089/) (2025)
- [SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Model](https://mlsanity.com/paper/2503.14463/) (2025)
- [Burst Super-Resolution with Diffusion Models for Improving Perceptual Quality](https://mlsanity.com/paper/2403.19428/) (2024)
- [Efficient Burst Super-Resolution with One-step Diffusion](https://mlsanity.com/paper/2507.13607/) (2025)
- [Optimized Vectorizing of Building Structures with Switch: High-Efficiency Convolutional Channel-Switch Hybridization Strategy](https://mlsanity.com/paper/2306.15035/) (2023)
- [Toward Accurate and Temporally Consistent Video Restoration from Raw Data](https://mlsanity.com/paper/2312.16247/) (2023)
- [Audio-Assisted Face Video Restoration with Temporal and Identity Complementary Learning](https://mlsanity.com/paper/2508.04161/) (2025)
- [MatRes: Zero-Shot Test-Time Model Adaptation for Simultaneous Matching and Restoration](https://mlsanity.com/paper/2604.10081/) (2026)

## Abstract (arXiv)

Denoising and demosaicking are two essential steps to reconstruct a clean full-color image from the raw data. Recently, joint denoising and demosaicking (JDD) for burst images, namely JDD-B, has attracted much attention by using multiple raw images captured in a short time to reconstruct a single high-quality image. One key challenge of JDD-B lies in the robust alignment of image frames. State-of-the-art alignment methods in feature domain cannot effectively utilize the temporal information of burst images, where large shifts commonly exist due to camera and object motion. In addition, the higher resolution (e.g., 4K) of modern imaging devices results in larger displacement between frames. To address these challenges, we design a differentiable two-stage alignment scheme sequentially in patch and pixel level for effective JDD-B. The input burst images are firstly aligned in the patch level by using a differentiable progressive block matching method, which can estimate the offset between distant frames with small computational cost. Then we perform implicit pixel-wise alignment in full-resolution feature domain to refine the alignment results. The two stages are jointly trained in an end-to-end manner. Extensive experiments demonstrate the significant improvement of our method over existing JDD-B methods. Codes are available at https://github.com/GuoShi28/2StageAlign.