Incentivizing honest performative predictions with proper scoring rules
Not on the map. Why?
Abstract (arXiv)
Proper scoring rules incentivize experts to accurately report beliefs, assuming predictions cannot influence outcomes. We relax this assumption and investigate incentives when predictions are performative, i.e., when they can influence the outcome of the prediction, such as when making public predictions about the stock market. We say a prediction is a fixed point if it accurately reflects the expert's beliefs after that prediction has been made. We show that in this setting, reports maximizing expected score generally do not reflect an expert's beliefs, and we give bounds on the inaccuracy of such reports. We show that, for binary predictions, if the influence of the expert's prediction on outcomes is bounded, it is possible to define scoring rules under which optimal reports are arbitrarily close to fixed points. However, this is impossible for predictions over more than two outcomes. We also perform numerical simulations in a toy setting, showing that our bounds are tight in some situations and that prediction error is often substantial (greater than 5-10%). Lastly, we discuss alternative notions of optimality, including performative stability, and show that they incentivize reporting fixed points.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 21 of its 57 references that are in the map. First posted on arXiv 28 May 2023.
Builds on
21 of 57 references.
References · 21
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2020Performative Predictioncited by 318
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
- 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
- 2019Optimal Policies Tend to Seek Powercited by 115
- 2021Truthful AI: Developing and governing AI that does not liecited by 109
- 2020Stochastic Optimization for Performative Predictioncited by 92
- 2021How to Learn when Data Reacts to Your Model: Performative Gradient Descentcited by 70
- 2018Stable Opponent Shaping in Differentiable Gamescited by 49
- 2018DiCE: The Infinitely Differentiable Monte-Carlo Estimatorcited by 46
- 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
9 more
- 2022Performative Powercited by 42
- 2022Regret Minimization with Performative Feedbackcited by 40
- 2023On Second-Order Scoring Rules for Epistemic Uncertainty Quantificationcited by 38
- 2020Optimization of Scoring Rulescited by 28
- 2017Good and safe uses of AI Oraclescited by 13
- 2020Binary Scoring Rules that Incentivize Precisioncited by 12
- 2020Avoiding Tampering Incentives in Deep RL via Decoupled Approvalcited by 12
- 2017Informational Substitutescited by 7
- 2022Scoring Rules for Performative Binary Predictioncited by 3
Built on by
Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 2.
9 papers.
- 2023Evaluating Superhuman Models with Consistency Checkscited by 47
- 2024Human Expertise in Algorithmic Predictioncited by 13
- 2025Revisiting the Predictability of Performative, Social Eventscited by 11
- 2025Statistical Inference under Performativitycited by 4
- 2026Dissecting Performative Prediction: A Comprehensive Surveycited by 2
- 2024Joint Scoring Rules: Zero-Sum Competition Avoids Performative Predictioncited by 1
- 2025Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictionscited by 1
- 2025Evolutionary Prediction Gamescited by 0
- 2026The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossiblecited by 0
- 2026The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossible25 May 2026
- 2026Dissecting Performative Prediction: A Comprehensive Survey10 Feb 2026
- 2025Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions24 Oct 2025
- 2025Statistical Inference under Performativity24 May 2025
- 2025Revisiting the Predictability of Performative, Social Events12 Mar 2025
- 2025Evolutionary Prediction Games5 Mar 2025
- 2024Joint Scoring Rules: Zero-Sum Competition Avoids Performative Prediction30 Dec 2024
- 2024Human Expertise in Algorithmic Prediction1 Feb 2024
- 2023Evaluating Superhuman Models with Consistency Checks16 Jun 2023