# Huge Ensembles Part I: Design of Ensemble Weather Forecasts using Spherical Fourier Neural Operators

arXiv: https://arxiv.org/abs/2408.03100 · HTML: https://mlsanity.com/paper/2408.03100/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [ACE: A fast, skillful learned global atmospheric model for climate prediction](https://mlsanity.com/paper/2310.02074/) (2023)
- [Advancing Parsimonious Deep Learning Weather Prediction using the HEALPix Mesh](https://mlsanity.com/paper/2311.06253/) (2023)
- [Uncertainty quantification for data-driven weather models](https://mlsanity.com/paper/2403.13458/) (2024)
- [A Practical Probabilistic Benchmark for AI Weather Models](https://mlsanity.com/paper/2401.15305/) (2024)
- [Joint Training of Deep Ensembles Fails Due to Learner Collusion](https://mlsanity.com/paper/2301.11323/) (2023)
- [Forecaster's Dilemma: Extreme Events and Forecast Evaluation](https://mlsanity.com/paper/1512.09244/) (2015)
- [Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction](https://mlsanity.com/paper/2404.19630/) (2024)
- [Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events](https://mlsanity.com/paper/2404.17652/) (2024)
- [FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting](https://mlsanity.com/paper/2405.05925/) (2024)
- [Huge Ensembles Part II: Properties of a Huge Ensemble of Hindcasts Generated with Spherical Fourier Neural Operators](https://mlsanity.com/paper/2408.01581/) (2024)
- [Evaluating forecasts for high-impact events using transformed kernel scores](https://mlsanity.com/paper/2202.12732/) (2022)
- [HEAL-ViT: Vision Transformers on a spherical mesh for medium-range weather forecasting](https://mlsanity.com/paper/2403.17016/) (2024)
- [Explaining the unexplainable: leveraging extremal dependence to characterize the 2021 Pacific Northwest heatwave](https://mlsanity.com/paper/2307.03688/) (2023)
- [Learning to forecast diagnostic parameters using pre-trained weather embedding](https://mlsanity.com/paper/2312.00290/) (2023)
- [A Machine Learning Outlook: Post-processing of Global Medium-range Forecasts](https://mlsanity.com/paper/2303.16301/) (2023)

## Built on by (most cited first, 18 of 18)

- [ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses](https://mlsanity.com/paper/2411.11268/) (2024)
- [Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling](https://mlsanity.com/paper/2408.10958/) (2024)
- [Samudra: An AI Global Ocean Emulator for Climate](https://mlsanity.com/paper/2412.03795/) (2024)
- [Can AI weather models predict out-of-distribution gray swan tropical cyclones?](https://mlsanity.com/paper/2410.14932/) (2024)
- [Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting](https://mlsanity.com/paper/2601.18111/) (2026)
- [Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning](https://mlsanity.com/paper/2506.10973/) (2025)
- [Binned Spectral Power Loss for Improved Prediction of Chaotic Systems](https://mlsanity.com/paper/2502.00472/) (2025)
- [Constructing Extreme Heatwave Storylines with Differentiable Climate Models](https://mlsanity.com/paper/2506.10660/) (2025)
- [Generalized Spherical Neural Operators: Green's Function Formulation](https://mlsanity.com/paper/2512.10723/) (2025)
- [Geometric Laplace Neural Operator](https://mlsanity.com/paper/2512.16409/) (2025)
- [Learning low-dimensional representations of ensemble forecast fields using autoencoder-based methods](https://mlsanity.com/paper/2502.04409/) (2025)
- [DEF: Diffusion-augmented Ensemble Forecasting](https://mlsanity.com/paper/2506.07324/) (2025)
- [On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification](https://mlsanity.com/paper/2511.17176/) (2025)
- [Neural Operators for Biomedical Spherical Heterogeneity](https://mlsanity.com/paper/2601.03561/) (2026)
- [Factorized Neural Operators Decompose Dynamic and Persistent Responses](https://mlsanity.com/paper/2606.16900/) (2026)
- [Rescene: band-limited stochastic forcing turns a frozen neural weather operator into a climate emulator](https://mlsanity.com/paper/2608.09971/) (2026)
- [Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Models](https://mlsanity.com/paper/2609.08412/) (2026)
- [Suitable Measures for the Potential Operational Utility of AI NWP Rainfall Forecasts Over Africa](https://mlsanity.com/paper/2609.31775/) (2026)

## Abstract (arXiv)

Studying low-likelihood high-impact extreme weather events in a warming world is a significant and challenging task for current ensemble forecasting systems. While these systems presently use up to 100 members, larger ensembles could enrich the sampling of internal variability. They may capture the long tails associated with climate hazards better than traditional ensemble sizes. Due to computational constraints, it is infeasible to generate huge ensembles (comprised of 1,000-10,000 members) with traditional, physics-based numerical models. In this two-part paper, we replace traditional numerical simulations with machine learning (ML) to generate hindcasts of huge ensembles. In Part I, we construct an ensemble weather forecasting system based on Spherical Fourier Neural Operators (SFNO), and we discuss important design decisions for constructing such an ensemble. The ensemble represents model uncertainty through perturbed-parameter techniques, and it represents initial condition uncertainty through bred vectors, which sample the fastest growing modes of the forecast. Using the European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (IFS) as a baseline, we develop an evaluation pipeline composed of mean, spectral, and extreme diagnostics. Using large-scale, distributed SFNOs with 1.1 billion learned parameters, we achieve calibrated probabilistic forecasts. As the trajectories of the individual members diverge, the ML ensemble mean spectra degrade with lead time, consistent with physical expectations. However, the individual ensemble members' spectra stay constant with lead time. Therefore, these members simulate realistic weather states, and the ML ensemble thus passes a crucial spectral test in the literature. The IFS and ML ensembles have similar Extreme Forecast Indices, and we show that the ML extreme weather forecasts are reliable and discriminating.