# Focal Frequency Loss for Image Reconstruction and Synthesis

arXiv: https://arxiv.org/abs/2012.12821 · HTML: https://mlsanity.com/paper/2012.12821/

## Key facts

- Cited by 204 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 23 Dec 2020.

## Built on by (most cited first, 30 of 204)

- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)
- [Intriguing properties of synthetic images: from generative adversarial networks to diffusion models](https://mlsanity.com/paper/2304.06408/) (2023)
- [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022)
- [Masked Frequency Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2206.07706/) (2022)
- [HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging](https://mlsanity.com/paper/2203.02149/) (2022)
- [FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information](https://mlsanity.com/paper/2308.03033/) (2023)
- [Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images](https://mlsanity.com/paper/2205.11733/) (2022)
- [Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining](https://mlsanity.com/paper/2404.01547/) (2024)
- [Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data](https://mlsanity.com/paper/2111.06849/) (2021)
- [TrustMark: Universal Watermarking for Arbitrary Resolution Images](https://mlsanity.com/paper/2311.18297/) (2023)
- [RDA: Robust Domain Adaptation via Fourier Adversarial Attacking](https://mlsanity.com/paper/2106.02874/) (2021)
- [CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding](https://mlsanity.com/paper/2304.09670/) (2023)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion](https://mlsanity.com/paper/2404.09140/) (2024)
- [Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution](https://mlsanity.com/paper/2405.07023/) (2024)
- [The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2204.08227/) (2022)
- [Architecture-Agnostic Masked Image Modeling -- From ViT back to CNN](https://mlsanity.com/paper/2205.13943/) (2022)
- [Explicitly Minimizing the Blur Error of Variational Autoencoders](https://mlsanity.com/paper/2304.05939/) (2023)
- [Frequency Domain-based Dataset Distillation](https://mlsanity.com/paper/2311.08819/) (2023)
- [FreeKD: Knowledge Distillation via Semantic Frequency Prompt](https://mlsanity.com/paper/2311.12079/) (2023)
- [SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution](https://mlsanity.com/paper/2312.05799/) (2023)
- [WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generation](https://mlsanity.com/paper/2207.07288/) (2022)
- [InvisMark: Invisible and Robust Watermarking for AI-generated Image Provenance](https://mlsanity.com/paper/2411.07795/) (2024)
- [The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding](https://mlsanity.com/paper/2512.19693/) (2025)
- [Y-Net: A Spatiospectral Dual-Encoder Networkfor Medical Image Segmentation](https://mlsanity.com/paper/2204.07613/) (2022)
- [Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.16484/) (2024)
- [Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detection](https://mlsanity.com/paper/2210.14485/) (2022)
- [Catch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder](https://mlsanity.com/paper/2305.02541/) (2023)
- [Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts](https://mlsanity.com/paper/2403.05026/) (2024)

## Abstract (arXiv)

Image reconstruction and synthesis have witnessed remarkable progress thanks to the development of generative models. Nonetheless, gaps could still exist between the real and generated images, especially in the frequency domain. In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further. We propose a novel focal frequency loss, which allows a model to adaptively focus on frequency components that are hard to synthesize by down-weighting the easy ones. This objective function is complementary to existing spatial losses, offering great impedance against the loss of important frequency information due to the inherent bias of neural networks. We demonstrate the versatility and effectiveness of focal frequency loss to improve popular models, such as VAE, pix2pix, and SPADE, in both perceptual quality and quantitative performance. We further show its potential on StyleGAN2.