# Learning in the Frequency Domain

arXiv: https://arxiv.org/abs/2002.12416 · HTML: https://mlsanity.com/paper/2002.12416/

## Key facts

- Cited by 198 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2020.

## Built on by (most cited first, 30 of 198)

- [FSDR: Frequency Space Domain Randomization for Domain Generalization](https://mlsanity.com/paper/2103.02370/) (2021)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [Improving Vision Transformers by Revisiting High-frequency Components](https://mlsanity.com/paper/2204.00993/) (2022)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [Parameter-Efficient Fine-Tuning with Discrete Fourier Transform](https://mlsanity.com/paper/2405.03003/) (2024)
- [SOLQ: Segmenting Objects by Learning Queries](https://mlsanity.com/paper/2106.02351/) (2021)
- [Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learning](https://mlsanity.com/paper/2112.13977/) (2021)
- [Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection](https://mlsanity.com/paper/2304.12489/) (2023)
- [RDA: Robust Domain Adaptation via Fourier Adversarial Attacking](https://mlsanity.com/paper/2106.02874/) (2021)
- [Spectral Unsupervised Domain Adaptation for Visual Recognition](https://mlsanity.com/paper/2106.06112/) (2021)
- [A Survey on Deep Learning based Time Series Analysis with Frequency Transformation](https://mlsanity.com/paper/2302.02173/) (2023)
- [Fighting deepfakes by detecting GAN DCT anomalies](https://mlsanity.com/paper/2101.09781/) (2021)
- [The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2204.08227/) (2022)
- [QARV: Quantization-Aware ResNet VAE for Lossy Image Compression](https://mlsanity.com/paper/2302.08899/) (2023)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain](https://mlsanity.com/paper/2207.07316/) (2022)
- [Frequency Domain-based Dataset Distillation](https://mlsanity.com/paper/2311.08819/) (2023)
- [FreeKD: Knowledge Distillation via Semantic Frequency Prompt](https://mlsanity.com/paper/2311.12079/) (2023)
- [FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning](https://mlsanity.com/paper/2312.04432/) (2023)
- [MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection](https://mlsanity.com/paper/2404.08452/) (2024)
- [Diverse Cotraining Makes Strong Semi-Supervised Segmentor](https://mlsanity.com/paper/2308.09281/) (2023)
- [Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention](https://mlsanity.com/paper/2405.06284/) (2024)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)
- [Privacy-Preserving Face Recognition Using Trainable Feature Subtraction](https://mlsanity.com/paper/2403.12457/) (2024)
- [Out-of-distribution Detection via Frequency-regularized Generative Models](https://mlsanity.com/paper/2208.09083/) (2022)
- [TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security](https://mlsanity.com/paper/2308.10133/) (2023)

## Abstract (arXiv)

Deep neural networks have achieved remarkable success in computer vision tasks. Existing neural networks mainly operate in the spatial domain with fixed input sizes. For practical applications, images are usually large and have to be downsampled to the predetermined input size of neural networks. Even though the downsampling operations reduce computation and the required communication bandwidth, it removes both redundant and salient information obliviously, which results in accuracy degradation. Inspired by digital signal processing theories, we analyze the spectral bias from the frequency perspective and propose a learning-based frequency selection method to identify the trivial frequency components which can be removed without accuracy loss. The proposed method of learning in the frequency domain leverages identical structures of the well-known neural networks, such as ResNet-50, MobileNetV2, and Mask R-CNN, while accepting the frequency-domain information as the input. Experiment results show that learning in the frequency domain with static channel selection can achieve higher accuracy than the conventional spatial downsampling approach and meanwhile further reduce the input data size. Specifically for ImageNet classification with the same input size, the proposed method achieves 1.41% and 0.66% top-1 accuracy improvements on ResNet-50 and MobileNetV2, respectively. Even with half input size, the proposed method still improves the top-1 accuracy on ResNet-50 by 1%. In addition, we observe a 0.8% average precision improvement on Mask R-CNN for instance segmentation on the COCO dataset.