# LoFormer: Local Frequency Transformer for Image Deblurring

arXiv: https://arxiv.org/abs/2407.16993 · HTML: https://mlsanity.com/paper/2407.16993/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 66 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Global Filter Networks for Image Classification](https://mlsanity.com/paper/2107.00645/) (2021)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Human-Aware Motion Deblurring](https://mlsanity.com/paper/2001.06816/) (2020)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [Stripformer: Strip Transformer for Fast Image Deblurring](https://mlsanity.com/paper/2204.04627/) (2022)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [Spectral Representations for Convolutional Neural Networks](https://mlsanity.com/paper/1506.03767/) (2015)

## Built on by (most cited first, 20 of 20)

- [From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspective](https://mlsanity.com/paper/2503.13165/) (2025)
- [MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration](https://mlsanity.com/paper/2503.09131/) (2025)
- [BlurDM: A Blur Diffusion Model for Image Deblurring](https://mlsanity.com/paper/2512.03979/) (2025)
- [XYScanNet: A State Space Model for Single Image Deblurring](https://mlsanity.com/paper/2412.10338/) (2024)
- [Scalable Event Cloud Network for Event-based Classification](https://mlsanity.com/paper/2412.20803/) (2024)
- [Reblurring-Guided Single Image Defocus Deblurring: A Learning Framework with Misaligned Training Pairs](https://mlsanity.com/paper/2409.17792/) (2024)
- [Efficient RAW Image Deblurring with Adaptive Frequency Modulation](https://mlsanity.com/paper/2505.24407/) (2025)
- [MBMamba: When Memory Buffer Meets Mamba for Structure-Aware Image Deblurring](https://mlsanity.com/paper/2508.12346/) (2025)
- [AIM 2025 Challenge on High FPS Motion Deblurring: Methods and Results](https://mlsanity.com/paper/2509.06793/) (2025)
- [MIORe & VAR-MIORe: Benchmarks to Push the Boundaries of Restoration](https://mlsanity.com/paper/2509.06803/) (2025)
- [SloMoDeblur: A Large-Scale Smartphone Image Deblurring Dataset](https://mlsanity.com/paper/2506.19445/) (2025)
- [Degradation-Aware Metric Prompting for Hyperspectral Image Restoration](https://mlsanity.com/paper/2512.20251/) (2025)
- [Unblur-SLAM: Dense Neural SLAM for Blurry Inputs](https://mlsanity.com/paper/2603.26810/) (2026)
- [Exploring Image Generation via Mutually Exclusive Probability Spaces and Local Correlation Hypothesis](https://mlsanity.com/paper/2506.21731/) (2025)
- [Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring](https://mlsanity.com/paper/2507.07708/) (2025)
- [RED: Robust Event-Guided Motion Deblurring with Modality-Specific Disentanglement](https://mlsanity.com/paper/2509.05554/) (2025)
- [Physically Interpretable Multi-Degradation Image Restoration via Deep Unfolding and Explainable Convolution](https://mlsanity.com/paper/2511.10166/) (2025)
- [DeepRFTv2: Kernel-level Learning for Image Deblurring](https://mlsanity.com/paper/2511.21132/) (2025)
- [Saturation-Aware Space-Variant Blind Image Deblurring](https://mlsanity.com/paper/2604.16200/) (2026)
- [UniDeblur: Unified Motion Deblurring via Fourier Convolution Learning](https://mlsanity.com/paper/2610.12941/) (2026)

## Abstract (arXiv)

Due to the computational complexity of self-attention (SA), prevalent techniques for image deblurring often resort to either adopting localized SA or employing coarse-grained global SA methods, both of which exhibit drawbacks such as compromising global modeling or lacking fine-grained correlation. In order to address this issue by effectively modeling long-range dependencies without sacrificing fine-grained details, we introduce a novel approach termed Local Frequency Transformer (LoFormer). Within each unit of LoFormer, we incorporate a Local Channel-wise SA in the frequency domain (Freq-LC) to simultaneously capture cross-covariance within low- and high-frequency local windows. These operations offer the advantage of (1) ensuring equitable learning opportunities for both coarse-grained structures and fine-grained details, and (2) exploring a broader range of representational properties compared to coarse-grained global SA methods. Additionally, we introduce an MLP Gating mechanism complementary to Freq-LC, which serves to filter out irrelevant features while enhancing global learning capabilities. Our experiments demonstrate that LoFormer significantly improves performance in the image deblurring task, achieving a PSNR of 34.09 dB on the GoPro dataset with 126G FLOPs. https://github.com/DeepMed-Lab-ECNU/Single-Image-Deblur