Truthful Elicitation of Imprecise Forecasts
Not on the map. Why?
Abstract (arXiv)
The quality of probabilistic forecasts is crucial for decision-making under uncertainty. While proper scoring rules incentivize truthful reporting of precise forecasts, they fall short when forecasters face epistemic uncertainty about their beliefs, limiting their use in safety-critical domains where decision-makers (DMs) prioritize proper uncertainty management. To address this, we propose a framework for scoring imprecise forecasts -- forecasts given as a set of beliefs. Despite existing impossibility results for deterministic scoring rules, we enable truthful elicitation by drawing connection to social choice theory and introducing a two-way communication framework where DMs first share their aggregation rules (e.g., averaging or min-max) used in downstream decisions for resolving forecast ambiguity. This, in turn, helps forecasters resolve indecision during elicitation. We further show that truthful elicitation of imprecise forecasts is achievable using proper scoring rules randomized over the aggregation procedure. Our approach allows DM to elicit and integrate the forecaster's epistemic uncertainty into their decision-making process, thus improving credibility.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 16 of its 58 references that are in the map. First posted on arXiv 20 Mar 2025.
Builds on
16 of 58 references.
References · 16
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
- 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
- 2021Domain Generalization: A Surveycited by 764
- 2013Domain Generalization via Invariant Feature Representationcited by 623
- 2021Omnipredictorscited by 74
- 2021Survival Regression with Proper Scoring Rules and Monotonic Neural Networkscited by 42
- 2023Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?cited by 40
- 2024Credal Learning Theorycited by 31
- 2024Credal Two-Sample Tests of Epistemic Uncertaintycited by 16
- 2024Domain Generalisation via Imprecise Learningcited by 14
- 2013Wind Energy: Forecasting Challenges for Its Operational Managementcited by 11
- 2024Scoring Rules and Calibration for Imprecise Probabilitiescited by 10
4 more
Built on by
Citing papers per year: 2025: 3; 2026: 3.
6 papers.
- 2025Integral Imprecise Probability Metricscited by 11
- 2026Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometrycited by 4
- 2025Property Elicitation on Imprecise Probabilitiescited by 3
- 2025When Do Credal Sets Stabilize? Fixed-Point Theorems for Credal Set Updatescited by 1
- 2026Incentive Aware AI Regulations: A Credal Characterisationcited by 0
- 2026How Much Imprecision is Enough Imprecision in my Classifier? A Practical Elicitation Procedurecited by 0
- 2026How Much Imprecision is Enough Imprecision in my Classifier? A Practical Elicitation Procedure27 Sep 2026
- 2026Incentive Aware AI Regulations: A Credal Characterisation5 Mar 2026
- 2026Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry2 Feb 2026
- 2025When Do Credal Sets Stabilize? Fixed-Point Theorems for Credal Set Updates6 Oct 2025
- 2025Property Elicitation on Imprecise Probabilities8 Jul 2025
- 2025Integral Imprecise Probability Metrics22 May 2025