# Uncertainty quantification for data-driven weather models

arXiv: https://arxiv.org/abs/2403.13458 · HTML: https://mlsanity.com/paper/2403.13458/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 57 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [Deep Learning for Post-Processing Ensemble Weather Forecasts](https://mlsanity.com/paper/2005.08748/) (2020)
- [AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning](https://mlsanity.com/paper/2308.13280/) (2023)
- [Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison](https://mlsanity.com/paper/2106.09512/) (2021)
- [SwinRDM: Integrate SwinRNN with Diffusion Model towards High-Resolution and High-Quality Weather Forecasting](https://mlsanity.com/paper/2306.03110/) (2023)
- [Isotonic Distributional Regression](https://mlsanity.com/paper/1909.03725/) (2019)
- [Generative machine learning methods for multivariate ensemble post-processing](https://mlsanity.com/paper/2211.01345/) (2022)
- [A Practical Probabilistic Benchmark for AI Weather Models](https://mlsanity.com/paper/2401.15305/) (2024)
- [Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling](https://mlsanity.com/paper/1302.7149/) (2013)
- [Forecaster's Dilemma: Extreme Events and Forecast Evaluation](https://mlsanity.com/paper/1512.09244/) (2015)
- [ENS-10: A Dataset For Post-Processing Ensemble Weather Forecasts](https://mlsanity.com/paper/2206.14786/) (2022)
- [Easy Uncertainty Quantification (EasyUQ): Generating Predictive Distributions from Single-valued Model Output](https://mlsanity.com/paper/2212.08376/) (2022)

## Built on by (most cited first, 26 of 26)

- [Probabilistic Weather Forecasting with Hierarchical Graph Neural Networks](https://mlsanity.com/paper/2406.04759/) (2024)
- [Huge Ensembles Part I: Design of Ensemble Weather Forecasts using Spherical Fourier Neural Operators](https://mlsanity.com/paper/2408.03100/) (2024)
- [Probabilistic neural operators for functional uncertainty quantification](https://mlsanity.com/paper/2502.12902/) (2025)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events](https://mlsanity.com/paper/2404.17652/) (2024)
- [Diffusion-LAM: Probabilistic Limited Area Weather Forecasting with Diffusion](https://mlsanity.com/paper/2502.07532/) (2025)
- [An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression](https://mlsanity.com/paper/2504.18433/) (2025)
- [CoDiCast: Conditional Diffusion Model for Global Weather Prediction with Uncertainty Quantification](https://mlsanity.com/paper/2409.05975/) (2024)
- [Split Conformal Prediction in the Function Space with Neural Operators](https://mlsanity.com/paper/2509.04623/) (2025)
- [Probabilistic measures afford fair comparisons of AIWP and NWP model output](https://mlsanity.com/paper/2506.03744/) (2025)
- [Improving Model Chain Approaches for Probabilistic Solar Energy Forecasting through Post-processing and Machine Learning](https://mlsanity.com/paper/2406.04424/) (2024)
- [Graph Neural Networks for Enhancing Ensemble Forecasts of Extreme Rainfall](https://mlsanity.com/paper/2504.05471/) (2025)
- [Uncertainty Quantification for Regression: A Unified Framework based on kernel scores](https://mlsanity.com/paper/2510.25599/) (2025)
- [Learning low-dimensional representations of ensemble forecast fields using autoencoder-based methods](https://mlsanity.com/paper/2502.04409/) (2025)
- [Dual-Splitting Conformal Prediction for Multi-Step Time Series Forecasting](https://mlsanity.com/paper/2503.21251/) (2025)
- [Improving Medium Range Severe Weather Prediction through Transformer Post-processing of AI Weather Forecasts](https://mlsanity.com/paper/2505.11750/) (2025)
- [A Geometry-Aware AI Emulator for the Coupled Whole Atmosphere from Earth Surface to the Ionosphere and Thermosphere](https://mlsanity.com/paper/2506.19340/) (2025)
- [On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification](https://mlsanity.com/paper/2511.17176/) (2025)
- [Efficient Credal Prediction through Decalibration](https://mlsanity.com/paper/2603.08495/) (2026)
- [Statistical post-processing yields accurate probabilistic forecasts from Artificial Intelligence weather models](https://mlsanity.com/paper/2504.12672/) (2025)
- [Breaking the Statistical Similarity Trap in Extreme Convection Detection](https://mlsanity.com/paper/2509.09195/) (2025)
- [Accurate, Efficient, and Explainable Deep Learning Approaches for Environmental Science Problems](https://mlsanity.com/paper/2605.19366/) (2026)
- [Can Deep Generative Models Reproduce Non-Stationary Gaussian Random Fields?](https://mlsanity.com/paper/2607.25929/) (2026)
- [Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributions](https://mlsanity.com/paper/2608.30795/) (2026)
- [Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Models](https://mlsanity.com/paper/2609.08412/) (2026)
- [Butterfly Effect and the Kinetic Energy Cascade in Probabilistic Machine Learning Weather Prediction Models](https://mlsanity.com/paper/2609.18489/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI)-based data-driven weather forecasting models have experienced rapid progress over the last years. Recent studies, with models trained on reanalysis data, achieve impressive results and demonstrate substantial improvements over state-of-the-art physics-based numerical weather prediction models across a range of variables and evaluation metrics. Beyond improved predictions, the main advantages of data-driven weather models are their substantially lower computational costs and the faster generation of forecasts, once a model has been trained. However, most efforts in data-driven weather forecasting have been limited to deterministic, point-valued predictions, making it impossible to quantify forecast uncertainties, which is crucial in research and for optimal decision making in applications. Our overarching aim is to systematically study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather. Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions, with the use of statistical and machine learning methods for post-hoc uncertainty quantification. In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results and provide improvements over ensemble forecasts from the physics-based ensemble weather model of the European Centre for Medium-Range Weather Forecasts for lead times of up to 5 days.