mlsanityData to 14 Oct 2026

Mitigating Spectral Bias in Neural Operators via High-Frequency Scaling for Physical Systems

arXiv 2503.13695 ↗First posted 17 Mar 2025cs.LG physics.comp-phcited by 31

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Neural operators have emerged as powerful surrogates for modeling complex physical problems. However, they suffer from spectral bias making them oblivious to high-frequency modes, which are present in multiscale physical systems. Therefore, they tend to produce over-smoothed solutions, which is particularly problematic in modeling turbulence and for systems with intricate patterns and sharp gradients such as multi-phase flow systems. In this work, we introduce a new approach named high-frequency scaling (HFS) to mitigate spectral bias in convolutional-based neural operators. By integrating HFS with proper variants of UNet neural operators, we demonstrate a higher prediction accuracy by mitigating spectral bias in single and two-phase flow problems. Unlike Fourier-based techniques, HFS is directly applied to the latent space, thus eliminating the computational cost associated with the Fourier transform. Additionally, we investigate alternative spectral bias mitigation through diffusion models conditioned on neural operators. While the diffusion model integrated with the standard neural operator may still suffer from significant errors, these errors are substantially reduced when the diffusion model is integrated with a HFS-enhanced neural operator.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 36 of its 66 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

36 of 66 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Generative AI for fast and accurate statistical computation of fluids27 Sep
  2. 2024Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling13 Sep
  3. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers10 Aug

Other references · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  9. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  10. 2018On the Spectral Bias of Neural Networkscited by 1,341
  11. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  12. 2023Symbolic Discovery of Optimization Algorithmscited by 434
21 more
  1. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  2. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 284
  3. 2022Transformer for Partial Differential Equations' Operator Learningcited by 283
  4. 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 261
  5. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 204
  6. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  7. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  8. 2022Learning two-phase microstructure evolution using neural operators and autoencoder architecturescited by 72
  9. 2022Blending Neural Operators and Relaxation Methods in PDE Numerical Solverscited by 54
  10. 2023Super-Resolution Neural Operatorcited by 49
  11. 2022Mitigating spectral bias for the multiscale operator learningcited by 44
  12. 2019A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problemscited by 35
  13. 2024RiemannONets: Interpretable Neural Operators for Riemann Problemscited by 28
  14. 2019Multi-scale Deep Neural Networks for Solving High Dimensional PDEscited by 25
  15. 2023Rethinking materials simulations: Blending direct numerical simulations with neural operatorscited by 25
  16. 2019The Spectral Bias of the Deep Image Priorcited by 21
  17. 2021Reproducing Activation Function for Deep Learningcited by 19
  18. 2023BubbleML: A Multi-Physics Dataset and Benchmarks for Machine Learningcited by 17
  19. 2023Real-time Inference and Extrapolation via a Diffusion-inspired Temporal Transformer Operator (DiTTO)cited by 16
  20. 2024Predicting Crack Nucleation and Propagation in Brittle Materials Using Deep Operator Networks with Diverse Trunk Architecturescited by 12
  21. 2025DeepVIVONet: Using deep neural operators to optimize sensor locations with application to vortex-induced vibrationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 17.

31 papers.

Sort:

By area:

  1. 2025FourierFlow: Frequency-aware Flow Matching for Generative Turbulence Modelingcited by 8
  2. 2025Learning Turbulent Flows with Generative Models: Super-resolution, Forecasting, and Sparse Flow Reconstructioncited by 8
  3. 2026Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelinescited by 8
  4. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  5. 2025A Physics-Informed Meta-Learning Framework for the Continuous Solution of Parametric PDEs on Arbitrary Geometriescited by 5
  6. 2025Bubbleformer: Forecasting Boiling with Transformerscited by 5
  7. 2025Stochastic and Non-local Closure Modeling for Nonlinear Dynamical Systems via Latent Score-based Generative Modelscited by 4
  8. 2026MENO: MeanFlow-Enhanced Neural Operators for Dynamical Systemscited by 3
  9. 2025Towards Generalizable PDE Dynamics Forecasting via Physics-Guided Invariant Learningcited by 2
  10. 2026Learning Physical Operators using Neural Operatorscited by 2
  11. 2026Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigationcited by 2
  12. 2025A Geometry-Aware AI Emulator for the Coupled Whole Atmosphere from Earth Surface to the Ionosphere and Thermospherecited by 1
19 more
  1. 2025Towards Robust Surrogate Models: Benchmarking Machine Learning Approaches to Expediting Phase Field Simulations of Brittle Fracturecited by 1
  2. 2025On the accuracy of implicit neural representations for cardiovascular anatomies and hemodynamic fieldscited by 1
  3. 2026Multi-Scale Wavelet Transformers for Operator Learning of Dynamical Systemscited by 1
  4. 2025Crack Path Prediction with Operator Learning using Discrete Particle System data Generationcited by 0
  5. 2025Learning Mappings in Mesh-based Simulationscited by 0
  6. 2025Analysis of Fourier Neural Operators via Effective Field Theorycited by 0
  7. 2025A Greedy PDE Router for Blending Neural Operators and Classical Methodscited by 0
  8. 2026The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositionscited by 0
  9. 2026UniFluids: Unified Neural Operator Learning with Conditional Flow-matchingcited by 0
  10. 2026GLU: Global-Local-Uncertainty Fusion for Scalable Spatiotemporal Reconstruction and Forecastingcited by 0
  11. 2026Autoregressive One-Step Generative Modeling for Dynamical System Forecastingcited by 0
  12. 2026UFO: A Domain-Unification-Free Operator Framework for Generalized Operator Learningcited by 0
  13. 2026NUCLEUS-MoE: Unified Model of Pool Boiling for Liquid Coolingcited by 0
  14. 2026(HB-ARFM) History-Bootstrapped Flow Matching for Inverse Boiling Reconstructioncited by 0
  15. 2026Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observationscited by 0
  16. 2026Read, Write, Relax: Why Neural PDE Surrogates Need Both Global and Local Processingcited by 0
  17. 2026Time series generation with spectrally aligned latent flow matchingcited by 0
  18. 2026Learn the Dynamics, Enforce the Physics: Generative Latent Operators for High-Dimensional Beam Evolutioncited by 0
  19. 2026Spectral Considerations on Unsupervised Time-domain EEG Representation Learning with Deep Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs
  2. 2025Resolving Turbulent Magnetohydrodynamics: A Hybrid Operator-Diffusion Framework
  3. 2025An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
  4. 2026Beyond the Manifold Hypothesis: Hybrid Spectral Parameterizations for Flow Matching
  5. 2026A Probabilistic Framework for Solving High-Frequency Helmholtz Equations via Diffusion Models
  6. 2026Learning to Discretize: Diffusion-Based Adaptive Mesh with Spectral Guidance
  7. 2023Turbulence Scaling from Deep Learning Diffusion Generative Models
  8. 2025Merging Memory and Space: A State Space Neural Operator