# Binned Spectral Power Loss for Improved Prediction of Chaotic Systems

arXiv: https://arxiv.org/abs/2502.00472 · HTML: https://mlsanity.com/paper/2502.00472/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 93 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Builds on (most cited first, 30 of 54)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks](https://mlsanity.com/paper/2006.13198/) (2020)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks](https://mlsanity.com/paper/1906.05747/) (2019)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [Learned Coarse Models for Efficient Turbulence Simulation](https://mlsanity.com/paper/2112.15275/) (2021)
- [Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains](https://mlsanity.com/paper/2007.11207/) (2020)
- [Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions](https://mlsanity.com/paper/2211.12316/) (2022)
- [On the Frequency Bias of Generative Models](https://mlsanity.com/paper/2111.02447/) (2021)
- [ACE: A fast, skillful learned global atmospheric model for climate prediction](https://mlsanity.com/paper/2310.02074/) (2023)
- [Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers](https://mlsanity.com/paper/2208.13273/) (2022)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [Training neural operators to preserve invariant measures of chaotic attractors](https://mlsanity.com/paper/2306.01187/) (2023)
- [Discount Factor as a Regularizer in Reinforcement Learning](https://mlsanity.com/paper/2007.02040/) (2020)
- [Mitigating spectral bias for the multiscale operator learning](https://mlsanity.com/paper/2210.10890/) (2022)
- [SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains](https://mlsanity.com/paper/2012.05535/) (2020)

## Built on by (most cited first, 10 of 10)

- [Learning Turbulent Flows with Generative Models: Super-resolution, Forecasting, and Sparse Flow Reconstruction](https://mlsanity.com/paper/2509.08752/) (2025)
- [Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelines](https://mlsanity.com/paper/2602.19265/) (2026)
- [Lazy Diffusion: Mitigating spectral collapse in generative diffusion-based stable autoregressive emulation of turbulent flows](https://mlsanity.com/paper/2512.09572/) (2025)
- [Deep Learning of Solver-Aware Turbulence Closures from Nudged LES Dynamics](https://mlsanity.com/paper/2604.23874/) (2026)
- [AI models of unstable flow exhibit hallucination](https://mlsanity.com/paper/2604.20372/) (2026)
- [Spectrally Regularized Latent Flow Matching for Turbulence Generation](https://mlsanity.com/paper/2606.11691/) (2026)
- [Multiscale Hypersonic Boundary Layer Reconstruction via Spectral Binning and Subdomain-wise Conditional Diffusion](https://mlsanity.com/paper/2606.15023/) (2026)
- [Scale-Aware Learning of Chaotic Dynamics on Unstructured Meshes via Binned Spectral Losses](https://mlsanity.com/paper/2607.19387/) (2026)
- [Improving Reduced-Order Rotating Detonation Engine Models with Data Assimilation and Machine Learning](https://mlsanity.com/paper/2609.16237/) (2026)
- [Staying on the Attractor: Supervising Neural Surrogates of 3D Turbulence Where They Leave It](https://mlsanity.com/paper/2609.32864/) (2026)

## Abstract (arXiv)

Forecasting multiscale chaotic dynamical systems, such as turbulent flows, with deep learning remains a formidable challenge due to the spectral bias of neural networks, which hinders the accurate representation of fine-scale structures in long-term predictions. This issue is exacerbated when models are deployed autoregressively, leading to compounding errors and instability. In this work, we introduce a novel approach to mitigate the spectral bias, which we call the Binned Spectral Power (BSP) Loss. The BSP loss is a frequency-domain loss function that adaptively weighs errors in predicting both larger and smaller scales of the dataset. Unlike traditional losses that focus on pointwise misfits, our BSP loss explicitly penalizes deviations in the energy distribution across different scales, promoting stable and physically consistent predictions. We demonstrate that the BSP loss mitigates the well-known problem of spectral bias in deep learning. We further validate our approach for the data-driven high-dimensional time-series forecasting of a range of benchmark chaotic systems, which are typically intractable due to spectral bias, culminating in experiments on canonical turbulent flow benchmarks. Our results demonstrate that the BSP loss significantly improves the stability and spectral accuracy of neural forecasting models without requiring architectural modifications. By directly targeting spectral consistency, our approach paves the way for more robust deep learning models for long-term forecasting of chaotic dynamical systems.