# BurstM: Deep Burst Multi-scale SR using Fourier Space with Optical Flow

arXiv: https://arxiv.org/abs/2409.15384 · HTML: https://mlsanity.com/paper/2409.15384/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 21 Sep 2024.

## Builds on (most cited first, 30 of 32)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art](https://mlsanity.com/paper/1704.05519/) (2017)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [Local Texture Estimator for Implicit Representation Function](https://mlsanity.com/paper/2111.08918/) (2021)
- [Deep Burst Super-Resolution](https://mlsanity.com/paper/2101.10997/) (2021)
- [Burst Image Restoration and Enhancement](https://mlsanity.com/paper/2110.03680/) (2021)
- [Deep Reparametrization of Multi-Frame Super-Resolution and Denoising](https://mlsanity.com/paper/2108.08286/) (2021)
- [BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment](https://mlsanity.com/paper/2204.08332/) (2022)
- [Burstormer: Burst Image Restoration and Enhancement Transformer](https://mlsanity.com/paper/2304.01194/) (2023)
- [NTIRE 2021 Challenge on Burst Super-Resolution: Methods and Results](https://mlsanity.com/paper/2106.03839/) (2021)
- [Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations](https://mlsanity.com/paper/2203.01325/) (2022)

## Built on by (most cited first, 7 of 7)

- [NTIRE 2025 Challenge on Efficient Burst HDR and Restoration: Datasets, Methods, and Results](https://mlsanity.com/paper/2505.12089/) (2025)
- [Benchmarking Burst Super-Resolution for Polarization Images: Noise Dataset and Analysis](https://mlsanity.com/paper/2503.18705/) (2025)
- [Feature Alignment with Equivariant Convolutions for Burst Image Super-Resolution](https://mlsanity.com/paper/2503.08300/) (2025)
- [Burst Image Super-Resolution via Multi-Cross Attention Encoding and Multi-Scan State-Space Decoding](https://mlsanity.com/paper/2505.19668/) (2025)
- [LatentBurst: A Fast and Efficient Multi Frame Super-Resolution for Hexadeca-Bayer Pattern CIS images](https://mlsanity.com/paper/2604.23268/) (2026)
- [The Need for Neural ISP in the Small-Pixel Era: How Shrinking Pixels Push Optics to the Limit and Neural Restoration Pushes Back](https://mlsanity.com/paper/2606.07675/) (2026)
- [Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset](https://mlsanity.com/paper/2607.04151/) (2026)

## Abstract (arXiv)

Multi frame super-resolution(MFSR) achieves higher performance than single image super-resolution (SISR), because MFSR leverages abundant information from multiple frames. Recent MFSR approaches adapt the deformable convolution network (DCN) to align the frames. However, the existing MFSR suffers from misalignments between the reference and source frames due to the limitations of DCN, such as small receptive fields and the predefined number of kernels. From these problems, existing MFSR approaches struggle to represent high-frequency information. To this end, we propose Deep Burst Multi-scale SR using Fourier Space with Optical Flow (BurstM). The proposed method estimates the optical flow offset for accurate alignment and predicts the continuous Fourier coefficient of each frame for representing high-frequency textures. In addition, we have enhanced the network flexibility by supporting various super-resolution (SR) scale factors with the unimodel. We demonstrate that our method has the highest performance and flexibility than the existing MFSR methods. Our source code is available at https://github.com/Egkang-Luis/burstm