# Bad and good errors: value-weighted skill scores in deep ensemble learning

arXiv: https://arxiv.org/abs/2103.02881 · HTML: https://mlsanity.com/paper/2103.02881/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 44 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Learning for IoT Big Data and Streaming Analytics: A Survey](https://mlsanity.com/paper/1712.04301/) (2017)
- [A hybrid supervised/unsupervised machine learning approach to solar flare prediction](https://mlsanity.com/paper/1706.07103/) (2017)
- [Machine learning as a flaring storm warning machine: Was a warning machine for the September 2017 solar flaring storm possible?](https://mlsanity.com/paper/2007.02425/) (2020)
- [Desaturating EUV observations of solar flaring storms](https://mlsanity.com/paper/1904.04211/) (2019)

## Built on by (most cited first, 5 of 5)

- [Score-oriented loss (SOL) functions](https://mlsanity.com/paper/2103.15522/) (2021)
- [Prediction of severe thunderstorm events with ensemble deep learning and radar data](https://mlsanity.com/paper/2109.09791/) (2021)
- [Operational solar flare forecasting via video-based deep learning](https://mlsanity.com/paper/2209.05128/) (2022)
- [A comprehensive theoretical framework for the optimization of neural networks classification performance with respect to weighted metrics](https://mlsanity.com/paper/2305.13472/) (2023)
- [Weighted Score-Oriented Losses for Temporally Localized Event Prediction](https://mlsanity.com/paper/2606.23145/) (2026)

## Abstract (arXiv)

In this paper we propose a novel approach to realize forecast verification. Specifically, we introduce a strategy for assessing the severity of forecast errors based on the evidence that, on the one hand, a false alarm just anticipating an occurring event is better than one in the middle of consecutive non-occurring events, and that, on the other hand, a miss of an isolated event has a worse impact than a miss of a single event, which is part of several consecutive occurrences. Relying on this idea, we introduce a novel definition of confusion matrix and skill scores giving greater importance to the value of the prediction rather than to its quality. Then, we introduce a deep ensemble learning procedure for binary classification, in which the probabilistic outcomes of a neural network are clustered via optimization of these value-weighted skill scores. We finally show the performances of this approach in the case of three applications concerned with pollution, space weather and stock prize forecasting.