# Forecast Aggregation via Peer Prediction

arXiv: https://arxiv.org/abs/1910.03779 · HTML: https://mlsanity.com/paper/1910.03779/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [Eliciting and Learning with Soft Labels from Every Annotator](https://mlsanity.com/paper/2207.00810/) (2022)
- [Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information](https://mlsanity.com/paper/2510.01499/) (2025)
- [Robust Decision Aggregation with Second-order Information](https://mlsanity.com/paper/2311.14094/) (2023)
- [Surprisingly Popular Voting Recovers Rankings, Surprisingly!](https://mlsanity.com/paper/2105.09386/) (2021)
- [Truthfulness Despite Weak Supervision: Evaluating and Training LLMs Using Peer Prediction](https://mlsanity.com/paper/2601.20299/) (2026)
- [Prior-Agnostic Robust Forecast Aggregation](https://mlsanity.com/paper/2604.24517/) (2026)
- [Sample Complexity of Forecast Aggregation](https://mlsanity.com/paper/2207.13126/) (2022)

## Abstract (arXiv)

Crowdsourcing enables the solicitation of forecasts on a variety of prediction tasks from distributed groups of people. How to aggregate the solicited forecasts, which may vary in quality, into an accurate final prediction remains a challenging yet critical question. Studies have found that weighing expert forecasts more in aggregation can improve the accuracy of the aggregated prediction. However, this approach usually requires access to the historical performance data of the forecasters, which are often not available. In this paper, we study the problem of aggregating forecasts without having historical performance data. We propose using peer prediction methods, a family of mechanisms initially designed to truthfully elicit private information in the absence of ground truth verification, to assess the expertise of forecasters, and then using this assessment to improve forecast aggregation. We evaluate our peer-prediction-aided aggregators on a diverse collection of 14 human forecast datasets. Compared with a variety of existing aggregators, our aggregators achieve a significant and consistent improvement on aggregation accuracy measured by the Brier score and the log score. Our results reveal the effectiveness of identifying experts to improve aggregation even without historical data.