# Optimization of Scoring Rules

arXiv: https://arxiv.org/abs/2007.02905 · HTML: https://mlsanity.com/paper/2007.02905/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 28 of 28)

- [U-Calibration: Forecasting for an Unknown Agent](https://mlsanity.com/paper/2307.00168/) (2023)
- [Contracting with a Learning Agent](https://mlsanity.com/paper/2401.16198/) (2024)
- [Eliciting Informative Text Evaluations with Large Language Models](https://mlsanity.com/paper/2405.15077/) (2024)
- [Calibration Error for Decision Making](https://mlsanity.com/paper/2404.13503/) (2024)
- [Can a calibration metric be both testable and actionable?](https://mlsanity.com/paper/2502.19851/) (2025)
- [Incentivizing honest performative predictions with proper scoring rules](https://mlsanity.com/paper/2305.17601/) (2023)
- [ElicitationGPT: Text Elicitation Mechanisms via Language Models](https://mlsanity.com/paper/2406.09363/) (2024)
- [Learning to Incentivize Information Acquisition: Proper Scoring Rules Meet Principal-Agent Model](https://mlsanity.com/paper/2303.08613/) (2023)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [A Perfectly Truthful Calibration Measure](https://mlsanity.com/paper/2508.13100/) (2025)
- [Optimal Scoring Rule Design under Partial Knowledge](https://mlsanity.com/paper/2107.07420/) (2021)
- [Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms](https://mlsanity.com/paper/2402.13567/) (2024)
- [Improved Bounds for Swap Multicalibration and Swap Omniprediction](https://mlsanity.com/paper/2505.20885/) (2025)
- [Calibration through the Lens of Indistinguishability](https://mlsanity.com/paper/2509.02279/) (2025)
- [Aligned Textual Scoring Rules](https://mlsanity.com/paper/2507.06221/) (2025)
- [Near-optimal Swap Regret Minimization for Convex Losses](https://mlsanity.com/paper/2602.08862/) (2026)
- [The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting](https://mlsanity.com/paper/2605.07671/) (2026)
- [Better Peer Grading through Bayesian Inference](https://mlsanity.com/paper/2209.01242/) (2022)
- [Underspecified Human Decision Experiments Considered Harmful](https://mlsanity.com/paper/2401.15106/) (2024)
- [Explanations are a Means to an End: Decision Theoretic Explanation Evaluation](https://mlsanity.com/paper/2506.22740/) (2025)
- [Score Design for Multi-Criteria Incentivization](https://mlsanity.com/paper/2410.06290/) (2024)
- [Provably Efficient Algorithm for Best Scoring Rule Identification in Online Principal-Agent Information Acquisition](https://mlsanity.com/paper/2505.17379/) (2025)
- [Designing DSIC Mechanisms for Data Sharing in the Era of Large Language Models](https://mlsanity.com/paper/2506.05379/) (2025)
- [Choice of Scoring Rules for Indirect Elicitation of Properties with Parametric Assumptions](https://mlsanity.com/paper/2506.17880/) (2025)
- [Next-Token Prediction and Regret Minimization](https://mlsanity.com/paper/2603.28499/) (2026)
- [When do prophets profit in prediction markets?](https://mlsanity.com/paper/2607.06166/) (2026)
- [Fast Rates for Swap-Agnostic Learning of Proper Losses](https://mlsanity.com/paper/2607.28856/) (2026)
- [Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics](https://mlsanity.com/paper/2608.01423/) (2026)

## Abstract (arXiv)

We characterize the optimal reward functions (scoring rules) that incentivize an agent to acquire information and report it truthfully to the principal. The optimal scoring rules let the agent make a simple binary bet in single-dimensional problems, and choose the dimension with the most surprising signal to be scored on in symmetric multi-dimensional problems. This scoring rule format remains approximately optimal for asymmetric distributions. These results demonstrate the importance of linking incentives to obtain high-quality information in multi-dimensional problems. In contrast, standard scoring rules like the quadratic scoring rule, or averages of single-dimensional scoring rules can be far from optimal.