# Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring

arXiv: https://arxiv.org/abs/2211.12250 · HTML: https://mlsanity.com/paper/2211.12250/

## Key facts

- Cited by 165 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Human-Aware Motion Deblurring](https://mlsanity.com/paper/2001.06816/) (2020)
- [Stripformer: Strip Transformer for Fast Image Deblurring](https://mlsanity.com/paper/2204.04627/) (2022)
- [Multi-class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02891/) (2022)
- [Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring](https://mlsanity.com/paper/2004.05343/) (2020)
- [Improving Image Restoration by Revisiting Global Information Aggregation](https://mlsanity.com/paper/2112.04491/) (2021)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [BANet: Blur-aware Attention Networks for Dynamic Scene Deblurring](https://mlsanity.com/paper/2101.07518/) (2021)
- [Physics-Based Generative Adversarial Models for Image Restoration and Beyond](https://mlsanity.com/paper/1808.00605/) (2018)

## Built on by (most cited first, 30 of 165)

- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [Hierarchical Integration Diffusion Model for Realistic Image Deblurring](https://mlsanity.com/paper/2305.12966/) (2023)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [DarkIR: Robust Low-Light Image Restoration](https://mlsanity.com/paper/2412.13443/) (2024)
- [BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling](https://mlsanity.com/paper/2403.04926/) (2024)
- [Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge](https://mlsanity.com/paper/2312.10299/) (2023)
- [Correlation Matching Transformation Transformers for UHD Image Restoration](https://mlsanity.com/paper/2406.00629/) (2024)
- [FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration](https://mlsanity.com/paper/2412.01427/) (2024)
- [DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models](https://mlsanity.com/paper/2407.01519/) (2024)
- [Efficient Visual State Space Model for Image Deblurring](https://mlsanity.com/paper/2405.14343/) (2024)
- [FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2401.03707/) (2024)
- [Learning Enriched Features via Selective State Spaces Model for Efficient Image Deblurring](https://mlsanity.com/paper/2403.20106/) (2024)
- [AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring](https://mlsanity.com/paper/2406.09135/) (2024)
- [Efficient Multi-scale Network with Learnable Discrete Wavelet Transform for Blind Motion Deblurring](https://mlsanity.com/paper/2401.00027/) (2023)
- [CU-Mamba: Selective State Space Models with Channel Learning for Image Restoration](https://mlsanity.com/paper/2404.11778/) (2024)
- [From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspective](https://mlsanity.com/paper/2503.13165/) (2025)
- [Prompt-based Ingredient-Oriented All-in-One Image Restoration](https://mlsanity.com/paper/2309.03063/) (2023)
- [Towards Real-world Event-guided Low-light Video Enhancement and Deblurring](https://mlsanity.com/paper/2408.14916/) (2024)
- [NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2504.12711/) (2025)
- [Effective Diffusion Transformer Architecture for Image Super-Resolution](https://mlsanity.com/paper/2409.19589/) (2024)
- [ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation](https://mlsanity.com/paper/2312.10998/) (2023)
- [Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring](https://mlsanity.com/paper/2404.13153/) (2024)
- [LoFormer: Local Frequency Transformer for Image Deblurring](https://mlsanity.com/paper/2407.16993/) (2024)
- [Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video](https://mlsanity.com/paper/2412.06424/) (2024)
- [Dropout the High-rate Downsampling: A Novel Design Paradigm for UHD Image Restoration](https://mlsanity.com/paper/2411.06456/) (2024)
- [Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration](https://mlsanity.com/paper/2404.00288/) (2024)
- [Instant GaussianImage: A Generalizable and Self-Adaptive Image Representation via 2D Gaussian Splatting](https://mlsanity.com/paper/2506.23479/) (2025)
- [Mixed Hierarchy Network for Image Restoration](https://mlsanity.com/paper/2302.09554/) (2023)
- [Multi-task Image Restoration Guided By Robust DINO Features](https://mlsanity.com/paper/2312.01677/) (2023)
- [Ultra-High-Definition Image Restoration: New Benchmarks and A Dual Interaction Prior-Driven Solution](https://mlsanity.com/paper/2406.13607/) (2024)

## Abstract (arXiv)

We present an effective and efficient method that explores the properties of Transformers in the frequency domain for high-quality image deblurring. Our method is motivated by the convolution theorem that the correlation or convolution of two signals in the spatial domain is equivalent to an element-wise product of them in the frequency domain. This inspires us to develop an efficient frequency domain-based self-attention solver (FSAS) to estimate the scaled dot-product attention by an element-wise product operation instead of the matrix multiplication in the spatial domain. In addition, we note that simply using the naive feed-forward network (FFN) in Transformers does not generate good deblurred results. To overcome this problem, we propose a simple yet effective discriminative frequency domain-based FFN (DFFN), where we introduce a gated mechanism in the FFN based on the Joint Photographic Experts Group (JPEG) compression algorithm to discriminatively determine which low- and high-frequency information of the features should be preserved for latent clear image restoration. We formulate the proposed FSAS and DFFN into an asymmetrical network based on an encoder and decoder architecture, where the FSAS is only used in the decoder module for better image deblurring. Experimental results show that the proposed method performs favorably against the state-of-the-art approaches. Code will be available at urlhttps://github.com/kkkls/FFTformer.