# Of Quantiles and Expectiles: Consistent Scoring Functions, Choquet Representations, and Forecast Rankings

arXiv: https://arxiv.org/abs/1503.08195 · HTML: https://mlsanity.com/paper/1503.08195/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2015.

## Built on by (most cited first, 28 of 28)

- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [Autocalibration and Tweedie-dominance for Insurance Pricing with Machine Learning](https://mlsanity.com/paper/2103.03635/) (2021)
- [Model Comparison and Calibration Assessment: User Guide for Consistent Scoring Functions in Machine Learning and Actuarial Practice](https://mlsanity.com/paper/2202.12780/) (2022)
- [Calibrating sufficiently](https://mlsanity.com/paper/2105.07283/) (2021)
- [Comparing Sequential Forecasters](https://mlsanity.com/paper/2110.00115/) (2021)
- [Evaluating Probabilistic Classifiers: The Triptych](https://mlsanity.com/paper/2301.10803/) (2023)
- [Neural networks for insurance pricing with frequency and severity data: a benchmark study from data preprocessing to technical tariff](https://mlsanity.com/paper/2310.12671/) (2023)
- [Sample-Efficient Omniprediction for Proper Losses](https://mlsanity.com/paper/2510.12769/) (2025)
- [Deep Huber quantile regression networks](https://mlsanity.com/paper/2306.10306/) (2023)
- [Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs](https://mlsanity.com/paper/2506.14540/) (2025)
- [Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces](https://mlsanity.com/paper/2609.37470/) (2026)
- [Conditionally Risk-Averse Contextual Bandits](https://mlsanity.com/paper/2210.13573/) (2022)
- [Time-Series Foundation AI Model for Value-at-Risk Forecasting](https://mlsanity.com/paper/2410.11773/) (2024)
- [Data-driven sparse identification of governing PDEs via knockoff filters and multi-criteria trade-offs](https://mlsanity.com/paper/2605.26631/) (2026)
- [On Learning the Tail Quantiles of Driving Behavior Distributions via Quantile Regression and Flows](https://mlsanity.com/paper/2305.13106/) (2023)
- [From Point to probabilistic gradient boosting for claim frequency and severity prediction](https://mlsanity.com/paper/2412.14916/) (2024)
- [Predicting clinical outcomes from patient care pathways represented with temporal knowledge graphs](https://mlsanity.com/paper/2502.21138/) (2025)
- [Forecastability as an Information-Theoretic Limit on Prediction](https://mlsanity.com/paper/2603.27074/) (2026)
- [Decision-Aligned Evaluation of Uncertainty Quantification](https://mlsanity.com/paper/2606.26990/) (2026)
- [Modeling High-Dimensional Data with Unknown Cut Points: A Fusion Penalized Logistic Threshold Regression](https://mlsanity.com/paper/2202.08441/) (2022)
- [Forecasting Local Ionospheric Parameters Using Transformers](https://mlsanity.com/paper/2502.15093/) (2025)
- [Variable transformations in consistent loss functions](https://mlsanity.com/paper/2502.16542/) (2025)
- [Explainable Boosting Machine for Predicting Claim Severity and Frequency in Car Insurance](https://mlsanity.com/paper/2503.21321/) (2025)
- [Neural ARFIMA model for forecasting BRIC exchange rates with long memory](https://mlsanity.com/paper/2509.06697/) (2025)
- [Quantifying the Risk-Return Tradeoff in Forecasting](https://mlsanity.com/paper/2605.09712/) (2026)
- [A prior-free blind detection of information leakage from model predictions](https://mlsanity.com/paper/2606.11267/) (2026)
- [Every pooling rule has its world: matching probability combination rules to situations and stakes](https://mlsanity.com/paper/2608.11275/) (2026)
- [Generating Benchmark Health Data Using a Tabular Diffusion Transformer](https://mlsanity.com/paper/2608.14496/) (2026)

## Abstract (arXiv)

In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be consistent for the functional at hand, in the sense that the expected score is minimized when following the directive. We show that any scoring function that is consistent for a quantile or an expectile functional, respectively, can be represented as a mixture of extremal scoring functions that form a linearly parameterized family. Scoring functions for the mean value and probability forecasts of binary events constitute important examples. The quantile and expectile functionals along with the respective extremal scoring functions admit appealing economic interpretations in terms of thresholds in decision making. The Choquet type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function. In empirical settings it suffices to compare the average scores for only a finite number of extremal elements. Plots of the average scores with respect to the extremal scoring functions, which we call Murphy diagrams, permit detailed comparisons of the relative merits of competing forecasts.