# Evaluating probabilistic forecasts of extremes using continuous ranked probability score distributions

arXiv: https://arxiv.org/abs/1905.04022 · HTML: https://mlsanity.com/paper/1905.04022/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2019.

## Built on by (most cited first, 7 of 7)

- [Precipitation Downscaling with Spatiotemporal Video Diffusion](https://mlsanity.com/paper/2312.06071/) (2023)
- [Enforcing tail calibration when training probabilistic forecast models](https://mlsanity.com/paper/2506.13687/) (2025)
- [Distributional regression and its evaluation with the CRPS: Bounds and convergence of the minimax risk](https://mlsanity.com/paper/2205.04360/) (2022)
- [Improving probabilistic forecasts of extreme wind speeds by training statistical post-processing models with weighted scoring rules](https://mlsanity.com/paper/2407.15900/) (2024)
- [Additive Covariance Matrix Models: Modelling Regional Electricity Net-Demand in Great Britain](https://mlsanity.com/paper/2211.07451/) (2022)
- [Generalisation Bounds of Zero-Shot Economic Forecasting using Time Series Foundation Models](https://mlsanity.com/paper/2506.15705/) (2025)
- [A Systematic Evaluation of Current Architectures in Wind Power Forecasting](https://mlsanity.com/paper/2606.02849/) (2026)

## Abstract (arXiv)

Verifying probabilistic forecasts for extreme events is a highly active research area because popular media and public opinions are naturally focused on extreme events, and biased conclusions are readily made. In this context, classical verification methods tailored for extreme events, such as thresholded and weighted scoring rules, have undesirable properties that cannot be mitigated, and the well-known continuous ranked probability score (CRPS) is no exception. In this paper, we define a formal framework for assessing the behavior of forecast evaluation procedures with respect to extreme events, which we use to demonstrate that assessment based on the expectation of a proper score is not suitable for extremes. Alternatively, we propose studying the properties of the CRPS as a random variable by using extreme value theory to address extreme event verification. An index is introduced to compare calibrated forecasts, which summarizes the ability of probabilistic forecasts for predicting extremes. The strengths and limitations of this method are discussed using both theoretical arguments and simulations.