mlsanityData to 7 Oct 2026

Long-Term Prediction Accuracy Improvement of Data-Driven Medium-Range Global Weather Forecast

arXiv 2407.01598 ↗First posted 26 Jun 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Long-term stability stands as a crucial requirement in data-driven medium-range global weather forecasting. Spectral bias is recognized as the primary contributor to instabilities, as data-driven methods difficult to learn small-scale dynamics. In this paper, we reveal that the universal mechanism for these instabilities is not only related to spectral bias but also to distortions brought by processing spherical data using conventional convolution. These distortions lead to a rapid amplification of errors over successive long-term iterations, resulting in a significant decline in forecast accuracy. To address this issue, a universal neural operator called the Spherical Harmonic Neural Operator (SHNO) is introduced to improve long-term iterative forecasts. SHNO uses the spherical harmonic basis to mitigate distortions for spherical data and uses gated residual spectral attention (GRSA) to correct spectral bias caused by spurious correlations across different scales. The effectiveness and merit of the proposed method have been validated through its application for spherical Shallow Water Equations (SWEs) and medium-range global weather forecasting. Our findings highlight the benefits and potential of SHNO to improve the accuracy of long-term prediction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

23 of 40 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2018On the Spectral Bias of Neural Networkscited by 1,321
  7. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  8. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  9. 2023ClimaX: A foundation model for weather and climatecited by 325
  10. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 225
  11. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
  12. 2023FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Leadcited by 170
11 more
  1. 2023FLatten Transformer: Vision Transformer using Focused Linear Attentioncited by 170
  2. 2020Improving data-driven global weather prediction using deep convolutional neural networks on a cubed spherecited by 95
  3. 2023Towards Stability of Autoregressive Neural Operatorscited by 54
  4. 2022Are Vision Transformers Robust to Spurious Correlations?cited by 36
  5. 2022Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inferencecited by 28
  6. 2021PDO-eS2CNNs: Partial Differential Operator Based Equivariant Spherical CNNscited by 19
  7. 2023Regularize implicit neural representation by itselfcited by 13
  8. 2019A Gated Self-attention Memory Network for Answer Selectioncited by 11
  9. 2022Adaptive and Implicit Regularization for Matrix Completioncited by 5
  10. 2024Reasoning in Transformers -- Mitigating Spurious Correlations and Reasoning Shortcutscited by 5
  11. 2024FViT: A Focal Vision Transformer with Gabor Filtercited by 3

Built on by

Not cited yet.