# Sample Complexity of Forecast Aggregation

arXiv: https://arxiv.org/abs/2207.13126 · HTML: https://mlsanity.com/paper/2207.13126/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [The Sample Complexity of Revenue Maximization](https://mlsanity.com/paper/1502.00963/) (2015)
- [Sample Complexity of Automated Mechanism Design](https://mlsanity.com/paper/1606.04145/) (2016)
- [Settling the Sample Complexity of Single-parameter Revenue Maximization](https://mlsanity.com/paper/1904.04962/) (2019)
- [Multi-Item Mechanisms without Item-Independence: Learnability via Robustness](https://mlsanity.com/paper/1911.02146/) (2019)
- [Generalizing Complex Hypotheses on Product Distributions: Auctions, Prophet Inequalities, and Pandora's Problem](https://mlsanity.com/paper/1911.11936/) (2019)
- [The Sample Complexity of Auctions with Side Information](https://mlsanity.com/paper/1511.02296/) (2015)
- [Learning Multi-item Auctions with (or without) Samples](https://mlsanity.com/paper/1709.00228/) (2017)
- [The Sample Complexity of Up-to-varepsilon Multi-Dimensional Revenue Maximization](https://mlsanity.com/paper/1808.02458/) (2018)
- [Are You Smarter Than a Random Expert? The Robust Aggregation of Substitutable Signals](https://mlsanity.com/paper/2111.03153/) (2021)
- [Elicitation for Aggregation](https://mlsanity.com/paper/1410.0375/) (2014)
- [Efficient Competitions and Online Learning with Strategic Forecasters](https://mlsanity.com/paper/2102.08358/) (2021)
- [The Wisdom of the Crowd and Higher-Order Beliefs](https://mlsanity.com/paper/2102.02666/) (2021)
- [Forecast Aggregation via Peer Prediction](https://mlsanity.com/paper/1910.03779/) (2019)
- [Eliciting Thinking Hierarchy without a Prior](https://mlsanity.com/paper/2109.10619/) (2021)
- [No-Regret Learning with Unbounded Losses: The Case of Logarithmic Pooling](https://mlsanity.com/paper/2202.11219/) (2022)
- [Learning of Optimal Forecast Aggregation in Partial Evidence Environments](https://mlsanity.com/paper/1802.07107/) (2018)

## Abstract (arXiv)

We consider a Bayesian forecast aggregation model where n experts, after observing private signals about an unknown binary event, report their posterior beliefs about the event to a principal, who then aggregates the reports into a single prediction for the event. The signals of the experts and the outcome of the event follow a joint distribution that is unknown to the principal, but the principal has access to i.i.d. "samples" from the distribution, where each sample is a tuple of the experts' reports (not signals) and the realization of the event. Using these samples, the principal aims to find an varepsilon-approximately optimal aggregator, where optimality is measured in terms of the expected squared distance between the aggregated prediction and the realization of the event. We show that the sample complexity of this problem is at least tilde Omega(mn-2 / varepsilon) for arbitrary discrete distributions, where m is the size of each expert's signal space. This sample complexity grows exponentially in the number of experts n. But, if the experts' signals are independent conditioned on the realization of the event, then the sample complexity is significantly reduced, to tilde O(1 / varepsilon2), which does not depend on n. Our results can be generalized to non-binary events. The proof of our results uses a reduction from the distribution learning problem and reveals the fact that forecast aggregation is almost as difficult as distribution learning.