When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study
Not on the map. Why?
Abstract (arXiv)
Partial differential equations (PDEs) often have high-frequency and multi-scale features that neural networks struggle to approximate. Physics-Informed Neural Networks (PINNs) build the governing equations directly into training, but suffer from spectral bias: they learn low-frequency components faster than high-frequency ones. Techniques such as Fourier feature embeddings and sinusoidal activations address this, but most studies assume they help across the board without checking which spectral regimes actually benefit. We introduce a dual-branch, spectrally-gated architecture (DBSG-PINN) that splits low- and high-frequency components into separate subnetworks joined by an adaptive gate, and use it to run a partially controlled ablation of frequency decomposition and spectral routing. We test this on five one-dimensional benchmark PDEs, ranging from smooth, single-scale problems to oscillatory, multi-scale ones. Frequency decomposition helps most on the spectrally complex benchmarks, cutting relative L2 error by up to 59.2% on a multimodal wave problem, but gives little benefit on smoother PDEs. On one benchmark (1D Wave), it performs substantially worse than a simpler fixed-combination variant. The gate's benefit scales with how spectrally rich the target solution is: the full model's advantage over the ablations is largest on multi-scale benchmarks and smallest (or negative) on single-scale ones, consistent with the gate exploiting frequency structure rather than acting as noise,though we do not directly visualize or quantify its spatial activations in this study. All results come from a single training seed across five 1D benchmarks, so we present this as an exploratory study meant to raise questions rather than answer them, and outline the additional seeds and benchmarks needed to test whether the pattern holds.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
17 of 32 references.
References · 17
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2015Automatic differentiation in machine learning: a surveycited by 846
- 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
- 2019DeepXDE: A deep learning library for solving differential equationscited by 617
- 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
5 more
- 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
- 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
- 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 142
- 2020Large-scale Neural Solvers for Partial Differential Equationscited by 17
- 2024Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learningcited by 9
Built on by
Not cited yet.