# Are You Smarter Than a Random Expert? The Robust Aggregation of Substitutable Signals

arXiv: https://arxiv.org/abs/2111.03153 · HTML: https://mlsanity.com/paper/2111.03153/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 33 references that are in the map.
- First posted on arXiv 4 Nov 2021.

## Builds on (most cited first, 3 of 3)

- [Elicitation for Aggregation](https://mlsanity.com/paper/1410.0375/) (2014)
- [Informational Substitutes](https://mlsanity.com/paper/1703.08636/) (2017)
- [Combining and Extremizing Real-Valued Forecasts](https://mlsanity.com/paper/1506.06405/) (2015)

## Built on by (most cited first, 13 of 13)

- [Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information](https://mlsanity.com/paper/2510.01499/) (2025)
- [AIA Forecaster: Technical Report](https://mlsanity.com/paper/2511.07678/) (2025)
- [Robust Decision Aggregation with Second-order Information](https://mlsanity.com/paper/2311.14094/) (2023)
- [Algorithmic Robust Forecast Aggregation](https://mlsanity.com/paper/2401.17743/) (2024)
- [The Surprising Benefits of Base Rate Neglect in Robust Aggregation](https://mlsanity.com/paper/2406.13490/) (2024)
- [Robust forecast aggregation via additional queries](https://mlsanity.com/paper/2512.05271/) (2025)
- [Robust Decision Aggregation with Adversarial Experts](https://mlsanity.com/paper/2403.08222/) (2024)
- [Tell Me Why: Incentivizing Explanations](https://mlsanity.com/paper/2502.13410/) (2025)
- [Prior-Agnostic Robust Forecast Aggregation](https://mlsanity.com/paper/2604.24517/) (2026)
- [Sample Complexity of Forecast Aggregation](https://mlsanity.com/paper/2207.13126/) (2022)
- [Mitigating the Participation Bias by Balancing Extreme Ratings](https://mlsanity.com/paper/2502.03737/) (2025)
- [Jailbreaking LLMs via Calibration](https://mlsanity.com/paper/2602.00619/) (2026)
- [Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts](https://mlsanity.com/paper/2609.33257/) (2026)

## Abstract (arXiv)

The problem of aggregating expert forecasts is ubiquitous in fields as wide-ranging as machine learning, economics, climate science, and national security. Despite this, our theoretical understanding of this question is fairly shallow. This paper initiates the study of forecast aggregation in a context where experts' knowledge is chosen adversarially from a broad class of information structures. While in full generality it is impossible to achieve a nontrivial performance guarantee, we show that doing so is possible under a condition on the experts' information structure that we call projective substitutes. The projective substitutes condition is a notion of informational substitutes: that there are diminishing marginal returns to learning the experts' signals. We show that under the projective substitutes condition, taking the average of the experts' forecasts improves substantially upon the strategy of trusting a random expert. We then consider a more permissive setting, in which the aggregator has access to the prior. We show that by averaging the experts' forecasts and then extremizing the average by moving it away from the prior by a constant factor, the aggregator's performance guarantee is substantially better than is possible without knowledge of the prior. Our results give a theoretical grounding to past empirical research on extremization and help give guidance on the appropriate amount to extremize.