# Extreme Weather Bench: A framework and benchmark for evaluation of high-impact weather

arXiv: https://arxiv.org/abs/2605.01126 · HTML: https://mlsanity.com/paper/2605.01126/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 72 references that are in the map.
- First posted on arXiv 1 May 2026.

## Builds on (most cited first, 10 of 10)

- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [Skillful joint probabilistic weather forecasting from marginals](https://mlsanity.com/paper/2506.10772/) (2025)
- [Dynamical Tests of a Deep-Learning Weather Prediction Model](https://mlsanity.com/paper/2309.10867/) (2023)
- [Forecaster's Dilemma: Extreme Events and Forecast Evaluation](https://mlsanity.com/paper/1512.09244/) (2015)
- [WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models](https://mlsanity.com/paper/2409.09371/) (2024)
- [Lightning-Fast Convective Outlooks: Predicting Severe Convective Environments with Global AI-based Weather Models](https://mlsanity.com/paper/2406.09474/) (2024)
- [Improving Medium Range Severe Weather Prediction through Transformer Post-processing of AI Weather Forecasts](https://mlsanity.com/paper/2505.11750/) (2025)

## Built on by (most cited first, 3 of 3)

- [4D Parallelism Unlocks Exascale Bayesian Neural Networks for High-Fidelity Atmospheric Modeling](https://mlsanity.com/paper/2609.12815/) (2026)
- [HazardWeaver: Scientific Route Selection for Hazard Analysis Agents](https://mlsanity.com/paper/2610.03591/) (2026)
- [Global Evaluation of AI and NWP Precipitation Forecasts During Atmospheric River Events](https://mlsanity.com/paper/2610.03758/) (2026)

## Abstract (arXiv)

Forecasting the wide variety of high-impact weather events experienced globally is a challenge for both Artificial Intelligence (AI) and Numerical Weather Prediction (NWP) models and it is critical that such models be properly verified before deployment. Although AI weather models are rapidly evolving, much of their evaluation is currently done either with a global-scale evaluation or by hand-picking a small number of case studies or a region. A widely-used open-source benchmark suite focusing on high-impact weather will help to drive the science forward for all scales of weather models, as it has for other AI fields. Here we introduce Extreme Weather Bench (EWB), a new community-driven benchmark suite that facilitates model validation and verification on a variety of high-impact hazards that matter to people around the globe. EWB provides a standard set of case studies (spanning across multiple spatial and temporal scales and different parts of the weather spectrum), observational data, impact-based metrics, and open-source code for users to evaluate their models. Verifying that a model works against a standard set of case studies, especially events that are high-impact for the general public, is a key piece of improving the trustworthiness of AI models. EWB will help to drive the science forward for all weather models, enabling true comparisons across models and evaluating models on specific high-impact phenomena through the use of case studies. EWB is a free open-source community-driven system and will continue to evolve to include additional phenomena, test cases and metrics in collaboration with the worldwide weather and forecast verification community.