# Proper scoring rules for estimation and forecast evaluation

arXiv: https://arxiv.org/abs/2504.01781 · HTML: https://mlsanity.com/paper/2504.01781/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 151 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 30 of 42)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [Score matching enables causal discovery of nonlinear additive noise models](https://mlsanity.com/paper/2203.04413/) (2022)
- [Statistical Efficiency of Score Matching: The View from Isoperimetry](https://mlsanity.com/paper/2210.00726/) (2022)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [A Spectral Energy Distance for Parallel Speech Synthesis](https://mlsanity.com/paper/2008.01160/) (2020)
- [Survival Regression with Proper Scoring Rules and Monotonic Neural Networks](https://mlsanity.com/paper/2103.14755/) (2021)
- [Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization](https://mlsanity.com/paper/2112.08217/) (2021)
- [Non-adversarial training of Neural SDEs with signature kernel scores](https://mlsanity.com/paper/2305.16274/) (2023)
- [Countdown Regression: Sharp and Calibrated Survival Predictions](https://mlsanity.com/paper/1806.08324/) (2018)
- [Isotonic Distributional Regression](https://mlsanity.com/paper/1909.03725/) (2019)
- [Generative machine learning methods for multivariate ensemble post-processing](https://mlsanity.com/paper/2211.01345/) (2022)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Score Matched Neural Exponential Families for Likelihood-Free Inference](https://mlsanity.com/paper/2012.10903/) (2020)
- [Estimation of High-Dimensional Graphical Models Using Regularized Score Matching](https://mlsanity.com/paper/1507.00433/) (2015)
- [Variational Inference with Gaussian Score Matching](https://mlsanity.com/paper/2307.07849/) (2023)
- [Variational Hamiltonian Monte Carlo via Score Matching](https://mlsanity.com/paper/1602.02219/) (2016)
- [Strictly proper kernel scores and characteristic kernels on compact spaces](https://mlsanity.com/paper/1712.05279/) (2017)
- [Regularizing with Bregman-Moreau envelopes](https://mlsanity.com/paper/1705.06019/) (2017)
- [Generalized Bayesian Likelihood-Free Inference](https://mlsanity.com/paper/2104.03889/) (2021)
- [Evaluating forecasts for high-impact events using transformed kernel scores](https://mlsanity.com/paper/2202.12732/) (2022)
- [Bayesian model comparison with the Hyv"arinen score: computation and consistency](https://mlsanity.com/paper/1711.00136/) (2017)
- [Proper Scoring Rules for Survival Analysis](https://mlsanity.com/paper/2305.00621/) (2023)
- [Decompositions of the mean continuous ranked probability score](https://mlsanity.com/paper/2311.14122/) (2023)
- [Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective](https://mlsanity.com/paper/1903.01944/) (2019)
- [Local scale invariance and robustness of proper scoring rules](https://mlsanity.com/paper/1912.05642/) (2019)
- [Comparing Sequential Forecasters](https://mlsanity.com/paper/2110.00115/) (2021)

## Built on by (most cited first, 18 of 18)

- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [Maximum Likelihood Reinforcement Learning](https://mlsanity.com/paper/2602.02710/) (2026)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [Probabilistic Retrofitting of Learned Simulators](https://mlsanity.com/paper/2603.01949/) (2026)
- [ScoringBench: A Benchmark for Evaluating Tabular Foundation Models with Proper Scoring Rules](https://mlsanity.com/paper/2603.29928/) (2026)
- [Enforcing tail calibration when training probabilistic forecast models](https://mlsanity.com/paper/2506.13687/) (2025)
- [Uncertainty Quantification for Regression using Proper Scoring Rules](https://mlsanity.com/paper/2509.26610/) (2025)
- [Generative Machine Learning for Multivariate Angular Simulation](https://mlsanity.com/paper/2504.21505/) (2025)
- [Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules](https://mlsanity.com/paper/2603.08206/) (2026)
- [Uncertainty Quantification for Regression: A Unified Framework based on kernel scores](https://mlsanity.com/paper/2510.25599/) (2025)
- [Generalisation Bounds of Zero-Shot Economic Forecasting using Time Series Foundation Models](https://mlsanity.com/paper/2506.15705/) (2025)
- [Multivariate Uncertainty Quantification with Tomographic Quantile Forests](https://mlsanity.com/paper/2512.16383/) (2025)
- [Prequential posteriors](https://mlsanity.com/paper/2511.17721/) (2025)
- [Forecasting Commencing Enrolments Under Data Sparsity: A Zero-Shot Time Series Foundation Models Framework for Higher Education Planning](https://mlsanity.com/paper/2602.12120/) (2026)
- [Probabilistic bias adjustment of seasonal forecasts using generative machine learning: A case study of Arctic sea ice predictions](https://mlsanity.com/paper/2605.29172/) (2026)
- [Censoring-Aware In-Context Learning for Generalized Supplier Lead Time Estimation in Supply Chain Planning](https://mlsanity.com/paper/2607.18530/) (2026)
- [Expressivity In Multimodal Contrastive Learning](https://mlsanity.com/paper/2608.17203/) (2026)
- [Variational Streaming Flow: Probabilistic Forecasting in Physical Time](https://mlsanity.com/paper/2610.00976/) (2026)

## Abstract (arXiv)

Proper scoring rules have been a subject of growing interest in recent years, not only as tools for evaluation of probabilistic forecasts but also as methods for estimating probability distributions. In this article, we review the mathematical foundations of proper scoring rules including general characterization results and important families of scoring rules. We discuss their role in statistics and machine learning for estimation and forecast evaluation. Furthermore, we comment on interesting developments of their usage in applications.