# Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems

arXiv: https://arxiv.org/abs/2503.13695 · HTML: https://mlsanity.com/paper/2503.13695/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 36 of its 66 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice](https://mlsanity.com/paper/2203.05962/) (2022)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Learning two-phase microstructure evolution using neural operators and autoencoder architectures](https://mlsanity.com/paper/2204.07230/) (2022)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [Super-Resolution Neural Operator](https://mlsanity.com/paper/2303.02584/) (2023)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling](https://mlsanity.com/paper/2409.08477/) (2024)
- [A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems](https://mlsanity.com/paper/1909.11759/) (2019)
- [Generative AI for fast and accurate statistical computation of fluids](https://mlsanity.com/paper/2409.18359/) (2024)
- [RiemannONets: Interpretable Neural Operators for Riemann Problems](https://mlsanity.com/paper/2401.08886/) (2024)
- [Multi-scale Deep Neural Networks for Solving High Dimensional PDEs](https://mlsanity.com/paper/1910.11710/) (2019)
- [Rethinking materials simulations: Blending direct numerical simulations with neural operators](https://mlsanity.com/paper/2312.05410/) (2023)

## Built on by (most cited first, 30 of 31)

- [FourierFlow: Frequency-aware Flow Matching for Generative Turbulence Modeling](https://mlsanity.com/paper/2506.00862/) (2025)
- [Learning Turbulent Flows with Generative Models: Super-resolution, Forecasting, and Sparse Flow Reconstruction](https://mlsanity.com/paper/2509.08752/) (2025)
- [Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelines](https://mlsanity.com/paper/2602.19265/) (2026)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [A Physics-Informed Meta-Learning Framework for the Continuous Solution of Parametric PDEs on Arbitrary Geometries](https://mlsanity.com/paper/2504.02459/) (2025)
- [Bubbleformer: Forecasting Boiling with Transformers](https://mlsanity.com/paper/2507.21244/) (2025)
- [Stochastic and Non-local Closure Modeling for Nonlinear Dynamical Systems via Latent Score-based Generative Models](https://mlsanity.com/paper/2506.20771/) (2025)
- [MENO: MeanFlow-Enhanced Neural Operators for Dynamical Systems](https://mlsanity.com/paper/2604.06881/) (2026)
- [Towards Generalizable PDE Dynamics Forecasting via Physics-Guided Invariant Learning](https://mlsanity.com/paper/2509.24332/) (2025)
- [Learning Physical Operators using Neural Operators](https://mlsanity.com/paper/2602.23113/) (2026)
- [Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation](https://mlsanity.com/paper/2605.24041/) (2026)
- [A Geometry-Aware AI Emulator for the Coupled Whole Atmosphere from Earth Surface to the Ionosphere and Thermosphere](https://mlsanity.com/paper/2506.19340/) (2025)
- [Towards Robust Surrogate Models: Benchmarking Machine Learning Approaches to Expediting Phase Field Simulations of Brittle Fracture](https://mlsanity.com/paper/2507.07237/) (2025)
- [On the accuracy of implicit neural representations for cardiovascular anatomies and hemodynamic fields](https://mlsanity.com/paper/2510.20970/) (2025)
- [Multi-Scale Wavelet Transformers for Operator Learning of Dynamical Systems](https://mlsanity.com/paper/2602.01486/) (2026)
- [Crack Path Prediction with Operator Learning using Discrete Particle System data Generation](https://mlsanity.com/paper/2506.01976/) (2025)
- [Learning Mappings in Mesh-based Simulations](https://mlsanity.com/paper/2506.12652/) (2025)
- [Analysis of Fourier Neural Operators via Effective Field Theory](https://mlsanity.com/paper/2507.21833/) (2025)
- [A Greedy PDE Router for Blending Neural Operators and Classical Methods](https://mlsanity.com/paper/2509.24814/) (2025)
- [The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions](https://mlsanity.com/paper/2602.21910/) (2026)
- [UniFluids: Unified Neural Operator Learning with Conditional Flow-matching](https://mlsanity.com/paper/2603.22309/) (2026)
- [GLU: Global-Local-Uncertainty Fusion for Scalable Spatiotemporal Reconstruction and Forecasting](https://mlsanity.com/paper/2603.26023/) (2026)
- [Autoregressive One-Step Generative Modeling for Dynamical System Forecasting](https://mlsanity.com/paper/2605.05540/) (2026)
- [UFO: A Domain-Unification-Free Operator Framework for Generalized Operator Learning](https://mlsanity.com/paper/2605.12700/) (2026)
- [NUCLEUS-MoE: Unified Model of Pool Boiling for Liquid Cooling](https://mlsanity.com/paper/2605.27722/) (2026)
- [(HB-ARFM) History-Bootstrapped Flow Matching for Inverse Boiling Reconstruction](https://mlsanity.com/paper/2606.00349/) (2026)
- [Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observations](https://mlsanity.com/paper/2606.03936/) (2026)
- [Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processing](https://mlsanity.com/paper/2608.21677/) (2026)
- [Time series generation with spectrally aligned latent flow matching](https://mlsanity.com/paper/2609.21989/) (2026)
- [Learn the Dynamics, Enforce the Physics: Generative Latent Operators for High-Dimensional Beam Evolution](https://mlsanity.com/paper/2610.14537/) (2026)

## Abstract (arXiv)

Neural operators have emerged as powerful surrogates for modeling complex physical problems. However, they suffer from spectral bias making them oblivious to high-frequency modes, which are present in multiscale physical systems. Therefore, they tend to produce over-smoothed solutions, which is particularly problematic in modeling turbulence and for systems with intricate patterns and sharp gradients such as multi-phase flow systems. In this work, we introduce a new approach named high-frequency scaling (HFS) to mitigate spectral bias in convolutional-based neural operators. By integrating HFS with proper variants of UNet neural operators, we demonstrate a higher prediction accuracy by mitigating spectral bias in single and two-phase flow problems. Unlike Fourier-based techniques, HFS is directly applied to the latent space, thus eliminating the computational cost associated with the Fourier transform. Additionally, we investigate alternative spectral bias mitigation through diffusion models conditioned on neural operators. While the diffusion model integrated with the standard neural operator may still suffer from significant errors, these errors are substantially reduced when the diffusion model is integrated with a HFS-enhanced neural operator.