# On the sensitivity of machine-learned probabilistic weather forecast models to scale-aware scoring rules

arXiv: https://arxiv.org/abs/2607.19161 · HTML: https://mlsanity.com/paper/2607.19161/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 9 of 9)

- [Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization](https://mlsanity.com/paper/2112.08217/) (2021)
- [Probabilistic neural operators for functional uncertainty quantification](https://mlsanity.com/paper/2502.12902/) (2025)
- [U-Cast: A Surprisingly Simple and Efficient Frontier Probabilistic AI Weather Forecaster](https://mlsanity.com/paper/2604.09041/) (2026)
- [EnScale: Temporally-consistent multivariate generative downscaling via proper scoring rules](https://mlsanity.com/paper/2509.26258/) (2025)
- [Proper Scoring Rules for Multivariate Probabilistic Forecasts based on Aggregation and Transformation](https://mlsanity.com/paper/2407.00650/) (2024)
- [A multi-scale loss formulation for learning a probabilistic model with proper score optimisation](https://mlsanity.com/paper/2506.10868/) (2025)
- [Learning Probabilistic Filters with Strictly Proper Scoring Rules](https://mlsanity.com/paper/2606.26497/) (2026)
- [(Sparse) Attention to the Details: Preserving Spectral Fidelity in ML-based Weather Forecasting Models](https://mlsanity.com/paper/2604.16429/) (2026)
- [A Composite-Loss Graph Neural Network for the Multivariate Post-Processing of Ensemble Weather Forecasts](https://mlsanity.com/paper/2509.02784/) (2025)

## Built on by (most cited first, 1 of 1)

- [Optimal Transport Dropout for Structured Predictive Uncertainty](https://mlsanity.com/paper/2609.33377/) (2026)

## Abstract (arXiv)

Probabilistic forecast models can be machine-learned from data using loss functions based on scoring rules such as the Continuous Ranked Probability Score (CRPS). This note summarises a preliminary study comparing versions of AIFS-CRPS, a global weather forecast model, trained with different univariate and multivariate scoring rules that aim to explicitly represent scale-awareness in the loss function. In the first part, we compare the (almost) fair CRPS, a fair global energy score, and a graph energy score based on node neighbourhoods. Across standard verification metrics, forecast skill is broadly similar. In the extratropics we find only small differences, while in the tropics the graph energy score setup performs somewhat better and the global energy score shows some degradation. These results suggest that multivariate scores are a viable alternative to CRPS-based training for global machine-learned weather forecasting. In the second part of the study, we analyse how different scoring rules and scale-aware loss constraints shape the spectra of forecast fields. It is apparent that any form of explicit scale-awareness improves realism. Here, the largest differences are likely associated with different effective weights per scale.