# An Imbalance-Robust Evaluation Framework for Extreme Risk Forecasts

arXiv: https://arxiv.org/abs/2512.00916 · HTML: https://mlsanity.com/paper/2512.00916/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 38 references that are in the map.
- First posted on arXiv 30 Nov 2025.

## Builds on (most cited first, 3 of 3)

- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [On Single Point Forecasts for Fat-Tailed Variables](https://mlsanity.com/paper/2007.16096/) (2020)
- [Behavior of prediction performance metrics with rare events](https://mlsanity.com/paper/2504.16185/) (2025)

## Built on by (most cited first, 1 of 1)

- [GeomHerd: A Forward-looking Herding Quantification via Ricci Flow Geometry on Agent Interactive Simulations](https://mlsanity.com/paper/2605.11645/) (2026)

## Abstract (arXiv)

Evaluating rare-event forecasts is challenging because standard metrics collapse as event prevalence declines. Measures such as F1-score, AUPRC, MCC, and accuracy induce degenerate thresholds -- converging to zero or one -- and their values become dominated by class imbalance rather than tail discrimination. We develop a family of rare-event-stable (RES) metrics whose optimal thresholds remain strictly interior as the event probability approaches zero, ensuring coherent decision rules under extreme rarity. Simulations spanning event probabilities from 0.01 down to one in a million show that RES metrics maintain stable thresholds, consistent model rankings, and near-complete prevalence invariance, whereas traditional metrics exhibit statistically significant threshold drift and structural collapse. A credit-default application confirms these results: RES metrics yield interpretable probability-of-default cutoffs (4-9%) and remain robust under subsampling, while classical metrics fail operationally. The RES framework provides a principled, prevalence-invariant basis for evaluating extreme-risk forecasts.