FourCastNeXt: Optimizing FourCastNet Training for Limited Compute
Not on the map. Why?
Abstract (arXiv)
FourCastNeXt is an optimization of FourCastNet - a global machine learning weather forecasting model - that performs with a comparable level of accuracy and can be trained using around 5% of the original FourCastNet computational requirements. This technical report presents strategies for model optimization that maintain similar performance as measured by the root-mean-square error (RMSE) of the modelled variables. By providing a model with very low comparative training costs, FourCastNeXt makes Neural Earth System Modelling much more accessible to researchers looking to conduct training experiments and ablation studies. FourCastNeXt training and inference code are available at https://github.com/nci/FourCastNeXt
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 10 Jan 2024.
Builds on
20 of 36 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2022A ConvNet for the 2020scited by 3,579
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020On Layer Normalization in the Transformer Architecturecited by 922
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
- 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
8 more
- 2023ClimaX: A foundation model for weather and climatecited by 337
- 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
- 2018An Empirical Model of Large-Batch Trainingcited by 281
- 2021Curriculum Learning: A Surveycited by 259
- 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
- 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 233
- 2022FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operatorscited by 188
- 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 178
Built on by
Citing papers per year: 2025: 1.
1 paper.