# Deep frequency principle towards understanding why deeper learning is faster

arXiv: https://arxiv.org/abs/2007.14313 · HTML: https://mlsanity.com/paper/2007.14313/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 30 of 31)

- [Overview frequency principle/spectral bias in deep learning](https://mlsanity.com/paper/2201.07395/) (2022)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [Towards Understanding the Condensation of Neural Networks at Initial Training](https://mlsanity.com/paper/2105.11686/) (2021)
- [Neural Networks Learn Statistics of Increasing Complexity](https://mlsanity.com/paper/2402.04362/) (2024)
- [What do neural networks learn in image classification? A frequency shortcut perspective](https://mlsanity.com/paper/2307.09829/) (2023)
- [Reconstruction from edge image combined with color and gradient difference for industrial surface anomaly detection](https://mlsanity.com/paper/2210.14485/) (2022)
- [Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width](https://mlsanity.com/paper/2205.12101/) (2022)
- [Loss Spike in Training Neural Networks](https://mlsanity.com/paper/2305.12133/) (2023)
- [Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs](https://mlsanity.com/paper/2112.06660/) (2021)
- [When Semantic Segmentation Meets Frequency Aliasing](https://mlsanity.com/paper/2403.09065/) (2024)
- [Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining](https://mlsanity.com/paper/2412.02454/) (2024)
- [Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space](https://mlsanity.com/paper/2402.12026/) (2024)
- [Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks](https://mlsanity.com/paper/2205.13283/) (2022)
- [Do ImageNet-trained models learn shortcuts? The impact of frequency shortcuts on generalization](https://mlsanity.com/paper/2503.03519/) (2025)
- [Frequency Principle in Deep Learning Beyond Gradient-descent-based Training](https://mlsanity.com/paper/2101.00747/) (2021)
- [Temporal Convolutional Explorer Helps Understand 1D-CNN's Learning Behavior in Time Series Classification from Frequency Domain](https://mlsanity.com/paper/2310.05467/) (2023)
- [Understanding the dynamics of the frequency bias in neural networks](https://mlsanity.com/paper/2405.14957/) (2024)
- [Butter: Frequency Consistency and Hierarchical Fusion for Autonomous Driving Object Detection](https://mlsanity.com/paper/2507.13373/) (2025)
- [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025)
- [Hypernym Bias: Unraveling Deep Classifier Training Dynamics through the Lens of Class Hierarchy](https://mlsanity.com/paper/2502.12125/) (2025)
- [Spatiotemporal wall pressure forecast of a rectangular cylinder with physics-aware DeepU-Fourier neural network](https://mlsanity.com/paper/2508.03183/) (2025)
- [FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformers](https://mlsanity.com/paper/2510.15385/) (2025)
- [ProtegoFed: Backdoor-Free Federated Instruction Tuning with Interspersed Poisoned Data](https://mlsanity.com/paper/2603.00516/) (2026)
- [FAIR: Frequency-aware Image Restoration for Industrial Visual Anomaly Detection](https://mlsanity.com/paper/2309.07068/) (2023)
- [Fast Fourier Correlation is a Highly Efficient and Accurate Feature Attribution Algorithm from the Perspective of Control Theory and Game Theory](https://mlsanity.com/paper/2504.02016/) (2025)
- [Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning](https://mlsanity.com/paper/2510.13865/) (2025)
- [Limitation of Characterizing Implicit Regularization by Data-independent Functions](https://mlsanity.com/paper/2201.12198/) (2022)
- [PWAVEP: Purifying Imperceptible Adversarial Perturbations in 3D Point Clouds via Spectral Graph Wavelets](https://mlsanity.com/paper/2602.03333/) (2026)
- [Gradient Residual Connections](https://mlsanity.com/paper/2602.09190/) (2026)
- [ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation](https://mlsanity.com/paper/2603.11498/) (2026)

## Abstract (arXiv)

Understanding the effect of depth in deep learning is a critical problem. In this work, we utilize the Fourier analysis to empirically provide a promising mechanism to understand why feedforward deeper learning is faster. To this end, we separate a deep neural network, trained by normal stochastic gradient descent, into two parts during analysis, i.e., a pre-condition component and a learning component, in which the output of the pre-condition one is the input of the learning one. We use a filtering method to characterize the frequency distribution of a high-dimensional function. Based on experiments of deep networks and real dataset, we propose a deep frequency principle, that is, the effective target function for a deeper hidden layer biases towards lower frequency during the training. Therefore, the learning component effectively learns a lower frequency function if the pre-condition component has more layers. Due to the well-studied frequency principle, i.e., deep neural networks learn lower frequency functions faster, the deep frequency principle provides a reasonable explanation to why deeper learning is faster. We believe these empirical studies would be valuable for future theoretical studies of the effect of depth in deep learning.