# Fourier Space Losses for Efficient Perceptual Image Super-Resolution

arXiv: https://arxiv.org/abs/2106.00783 · HTML: https://mlsanity.com/paper/2106.00783/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Lightweight Image Super-Resolution with Information Multi-distillation Network](https://mlsanity.com/paper/1909.11856/) (2019)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [The 2018 PIRM Challenge on Perceptual Image Super-resolution](https://mlsanity.com/paper/1809.07517/) (2018)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Learning a No-Reference Quality Metric for Single-Image Super-Resolution](https://mlsanity.com/paper/1612.05890/) (2016)
- [Frequency Separation for Real-World Super-Resolution](https://mlsanity.com/paper/1911.07850/) (2019)
- [AutoGAN: Neural Architecture Search for Generative Adversarial Networks](https://mlsanity.com/paper/1908.03835/) (2019)
- [NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01996/) (2020)
- [AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks](https://mlsanity.com/paper/2006.08198/) (2020)
- [AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results](https://mlsanity.com/paper/2009.06943/) (2020)
- [Wavelet Domain Style Transfer for an Effective Perception-distortion Tradeoff in Single Image Super-Resolution](https://mlsanity.com/paper/1910.04074/) (2019)
- [Journey Towards Tiny Perceptual Super-Resolution](https://mlsanity.com/paper/2007.04356/) (2020)
- [NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results](https://mlsanity.com/paper/2005.01056/) (2020)
- [AIM 2019 Challenge on Constrained Super-Resolution: Methods and Results](https://mlsanity.com/paper/1911.01249/) (2019)

## Built on by (most cited first, 30 of 73)

- [Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution](https://mlsanity.com/paper/2203.09195/) (2022)
- [Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2203.14216/) (2022)
- [FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information](https://mlsanity.com/paper/2308.03033/) (2023)
- [DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models](https://mlsanity.com/paper/2307.02457/) (2023)
- [DocDiff: Document Enhancement via Residual Diffusion Models](https://mlsanity.com/paper/2305.03892/) (2023)
- [FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining](https://mlsanity.com/paper/2405.19450/) (2024)
- [Visual Fourier Prompt Tuning](https://mlsanity.com/paper/2411.01327/) (2024)
- [A Systematic Survey of Deep Learning-based Single-Image Super-Resolution](https://mlsanity.com/paper/2109.14335/) (2021)
- [ARM: Any-Time Super-Resolution Method](https://mlsanity.com/paper/2203.10812/) (2022)
- [LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2308.11116/) (2023)
- [OPE-SR: Orthogonal Position Encoding for Designing a Parameter-free Upsampling Module in Arbitrary-scale Image Super-Resolution](https://mlsanity.com/paper/2303.01091/) (2023)
- [Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts](https://mlsanity.com/paper/2402.19215/) (2024)
- [Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcasting](https://mlsanity.com/paper/2410.23159/) (2024)
- [DMFourLLIE: Dual-Stage and Multi-Branch Fourier Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2412.00683/) (2024)
- [Frequency-Adaptive Pan-Sharpening with Mixture of Experts](https://mlsanity.com/paper/2401.02151/) (2024)
- [Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs](https://mlsanity.com/paper/2409.17778/) (2024)
- [Generative Adversarial Networks for Image Super-Resolution: A Survey](https://mlsanity.com/paper/2204.13620/) (2022)
- [Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspective](https://mlsanity.com/paper/2312.15408/) (2023)
- [Training Transformer Models by Wavelet Losses Improves Quantitative and Visual Performance in Single Image Super-Resolution](https://mlsanity.com/paper/2404.11273/) (2024)
- [Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting](https://mlsanity.com/paper/2401.10564/) (2024)
- [Rethinking Brain Tumor Segmentation from the Frequency Domain Perspective](https://mlsanity.com/paper/2506.10142/) (2025)
- [On the Effectiveness of Spectral Discriminators for Perceptual Quality Improvement](https://mlsanity.com/paper/2307.12027/) (2023)
- [Spatial-frequency Dual-Domain Feature Fusion Network for Low-Light Remote Sensing Image Enhancement](https://mlsanity.com/paper/2404.17400/) (2024)
- [Learning Multi-scale Spatial-frequency Features for Image Denoising](https://mlsanity.com/paper/2506.16307/) (2025)
- [DSRGAN: Detail Prior-Assisted Perceptual Single Image Super-Resolution via Generative Adversarial Networks](https://mlsanity.com/paper/2112.13191/) (2021)
- [Spatial-Frequency Attention for Image Denoising](https://mlsanity.com/paper/2302.13598/) (2023)
- [RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing](https://mlsanity.com/paper/2405.09083/) (2024)
- [Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution](https://mlsanity.com/paper/2507.14367/) (2025)
- [RPBG: Towards Robust Neural Point-based Graphics in the Wild](https://mlsanity.com/paper/2405.05663/) (2024)
- [VSRM: A Robust Mamba-Based Framework for Video Super-Resolution](https://mlsanity.com/paper/2506.22762/) (2025)

## Abstract (arXiv)

Many super-resolution (SR) models are optimized for high performance only and therefore lack efficiency due to large model complexity. As large models are often not practical in real-world applications, we investigate and propose novel loss functions, to enable SR with high perceptual quality from much more efficient models. The representative power for a given low-complexity generator network can only be fully leveraged by strong guidance towards the optimal set of parameters. We show that it is possible to improve the performance of a recently introduced efficient generator architecture solely with the application of our proposed loss functions. In particular, we use a Fourier space supervision loss for improved restoration of missing high-frequency (HF) content from the ground truth image and design a discriminator architecture working directly in the Fourier domain to better match the target HF distribution. We show that our losses' direct emphasis on the frequencies in Fourier-space significantly boosts the perceptual image quality, while at the same time retaining high restoration quality in comparison to previously proposed loss functions for this task. The performance is further improved by utilizing a combination of spatial and frequency domain losses, as both representations provide complementary information during training. On top of that, the trained generator achieves comparable results with and is 2.4x and 48x faster than state-of-the-art perceptual SR methods RankSRGAN and SRFlow respectively.