# Truthful Elicitation of Imprecise Forecasts

arXiv: https://arxiv.org/abs/2503.16395 · HTML: https://mlsanity.com/paper/2503.16395/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Omnipredictors](https://mlsanity.com/paper/2109.05389/) (2021)
- [Survival Regression with Proper Scoring Rules and Monotonic Neural Networks](https://mlsanity.com/paper/2103.14755/) (2021)
- [Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?](https://mlsanity.com/paper/2306.09586/) (2023)
- [Credal Learning Theory](https://mlsanity.com/paper/2402.00957/) (2024)
- [Credal Two-Sample Tests of Epistemic Uncertainty](https://mlsanity.com/paper/2410.12921/) (2024)
- [Domain Generalisation via Imprecise Learning](https://mlsanity.com/paper/2404.04669/) (2024)
- [Wind Energy: Forecasting Challenges for Its Operational Management](https://mlsanity.com/paper/1312.6471/) (2013)
- [Scoring Rules and Calibration for Imprecise Probabilities](https://mlsanity.com/paper/2410.23001/) (2024)
- [Proper Scoring Rules for Survival Analysis](https://mlsanity.com/paper/2305.00621/) (2023)
- [ElicitationGPT: Text Elicitation Mechanisms via Language Models](https://mlsanity.com/paper/2406.09363/) (2024)
- [Language Generation with Strictly Proper Scoring Rules](https://mlsanity.com/paper/2405.18906/) (2024)
- [Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections](https://mlsanity.com/paper/2307.11672/) (2023)

## Built on by (most cited first, 6 of 6)

- [Integral Imprecise Probability Metrics](https://mlsanity.com/paper/2505.16156/) (2025)
- [Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry](https://mlsanity.com/paper/2602.01667/) (2026)
- [Property Elicitation on Imprecise Probabilities](https://mlsanity.com/paper/2507.05857/) (2025)
- [When Do Credal Sets Stabilize? Fixed-Point Theorems for Credal Set Updates](https://mlsanity.com/paper/2510.04769/) (2025)
- [Incentive Aware AI Regulations: A Credal Characterisation](https://mlsanity.com/paper/2603.05175/) (2026)
- [How Much Imprecision is Enough Imprecision in my Classifier? A Practical Elicitation Procedure](https://mlsanity.com/paper/2609.33352/) (2026)

## Abstract (arXiv)

The quality of probabilistic forecasts is crucial for decision-making under uncertainty. While proper scoring rules incentivize truthful reporting of precise forecasts, they fall short when forecasters face epistemic uncertainty about their beliefs, limiting their use in safety-critical domains where decision-makers (DMs) prioritize proper uncertainty management. To address this, we propose a framework for scoring imprecise forecasts -- forecasts given as a set of beliefs. Despite existing impossibility results for deterministic scoring rules, we enable truthful elicitation by drawing connection to social choice theory and introducing a two-way communication framework where DMs first share their aggregation rules (e.g., averaging or min-max) used in downstream decisions for resolving forecast ambiguity. This, in turn, helps forecasters resolve indecision during elicitation. We further show that truthful elicitation of imprecise forecasts is achievable using proper scoring rules randomized over the aggregation procedure. Our approach allows DM to elicit and integrate the forecaster's epistemic uncertainty into their decision-making process, thus improving credibility.