# Flow-Guided Sparse Transformer for Video Deblurring

arXiv: https://arxiv.org/abs/2201.01893 · HTML: https://mlsanity.com/paper/2201.01893/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 58 references that are in the map.
- First posted on arXiv 6 Jan 2022.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Scaling Local Self-Attention for Parameter Efficient Visual Backbones](https://mlsanity.com/paper/2103.12731/) (2021)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)

## Built on by (most cited first, 30 of 43)

- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2203.04845/) (2022)
- [HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging](https://mlsanity.com/paper/2203.02149/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation](https://mlsanity.com/paper/2305.15302/) (2023)
- [Learning the Degradation Distribution for Blind Image Super-Resolution](https://mlsanity.com/paper/2203.04962/) (2022)
- [Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior](https://mlsanity.com/paper/2303.09757/) (2023)
- [On the Generalization of BasicVSR++ to Video Deblurring and Denoising](https://mlsanity.com/paper/2204.05308/) (2022)
- [Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring](https://mlsanity.com/paper/2406.07551/) (2024)
- [Mixed Hierarchy Network for Image Restoration](https://mlsanity.com/paper/2302.09554/) (2023)
- [Learning Truncated Causal History Model for Video Restoration](https://mlsanity.com/paper/2410.03936/) (2024)
- [Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration](https://mlsanity.com/paper/2205.10195/) (2022)
- [Rethinking Video Deblurring with Wavelet-Aware Dynamic Transformer and Diffusion Model](https://mlsanity.com/paper/2408.13459/) (2024)
- [CMTA: Cross-Modal Temporal Alignment for Event-guided Video Deblurring](https://mlsanity.com/paper/2408.14930/) (2024)
- [Domain-adaptive Video Deblurring via Test-time Blurring](https://mlsanity.com/paper/2407.09059/) (2024)
- [Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision](https://mlsanity.com/paper/2506.06253/) (2025)
- [NTIRE 2023 Quality Assessment of Video Enhancement Challenge](https://mlsanity.com/paper/2307.09729/) (2023)
- [Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos](https://mlsanity.com/paper/2309.08022/) (2023)
- [Event-Enhanced Blurry Video Super-Resolution](https://mlsanity.com/paper/2504.13042/) (2025)
- [EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation](https://mlsanity.com/paper/2506.03512/) (2025)
- [Decoupling Degradations with Recurrent Network for Video Restoration in Under-Display Camera](https://mlsanity.com/paper/2403.05660/) (2024)
- [Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement](https://mlsanity.com/paper/2202.00011/) (2022)
- [Spatial Feature Mapping for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2206.01831/) (2022)
- [Reference-based Restoration of Digitized Analog Videotapes](https://mlsanity.com/paper/2310.14926/) (2023)
- [DAVIDE: Depth-Aware Video Deblurring](https://mlsanity.com/paper/2409.01274/) (2024)
- [A Survey: Spatiotemporal Consistency in Video Generation](https://mlsanity.com/paper/2502.17863/) (2025)

## Abstract (arXiv)

Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring. However, CNN-based methods show limitations in capturing long-range dependencies and modeling non-local self-similarity. In this paper, we propose a novel framework, Flow-Guided Sparse Transformer (FGST), for video deblurring. In FGST, we customize a self-attention module, Flow-Guided Sparse Window-based Multi-head Self-Attention (FGSW-MSA). For each query element on the blurry reference frame, FGSW-MSA enjoys the guidance of the estimated optical flow to globally sample spatially sparse yet highly related key elements corresponding to the same scene patch in neighboring frames. Besides, we present a Recurrent Embedding (RE) mechanism to transfer information from past frames and strengthen long-range temporal dependencies. Comprehensive experiments demonstrate that our proposed FGST outperforms state-of-the-art (SOTA) methods on both DVD and GOPRO datasets and even yields more visually pleasing results in real video deblurring. Code and pre-trained models are publicly available at https://github.com/linjing7/VR-Baseline