# Evaluating Strategic Reasoning in Forecasting Agents

arXiv: https://arxiv.org/abs/2604.26106 · HTML: https://mlsanity.com/paper/2604.26106/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Builds on (most cited first, 7 of 7)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Forecasting Future World Events with Neural Networks](https://mlsanity.com/paper/2206.15474/) (2022)
- [LLM-as-a-Prophet: Understanding Predictive Intelligence with Prophet Arena](https://mlsanity.com/paper/2510.17638/) (2025)
- [Deep Research Bench: Evaluating AI Web Research Agents](https://mlsanity.com/paper/2506.06287/) (2025)
- [Bench to the Future: A Pastcasting Benchmark for Forecasting Agents](https://mlsanity.com/paper/2506.21558/) (2025)
- [Automating Forecasting Question Generation and Resolution for AI Evaluation](https://mlsanity.com/paper/2601.22444/) (2026)

## Built on by (most cited first, 1 of 1)

- [Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agents](https://mlsanity.com/paper/2609.28876/) (2026)

## Abstract (arXiv)

Forecasting benchmarks produce accuracy leaderboards but little insight into why some forecasters are more accurate than others. We introduce Bench to the Future 2 (BTF-2), 1,417 pastcasting questions with a frozen 15M-document research corpus in which agents reproducibly research and forecast offline, producing full reasoning traces. BTF-2 detects accuracy differences of 0.004 Brier score, and can distinguish differential agent strengths in research vs. judgment. We build a forecaster 0.011 Brier more accurate than any single frontier agent, and use it to evaluate agent strategic reasoning without hindsight bias. We find the better forecaster differs primarily in its pre-mortem analysis of its blind spots and consideration of black swans. Expert human forecasters found the dominant strategic reasoning failures of frontier agents are in assessing political and business leaders' incentives, judging their likelihood to follow through on stated plans, and modeling institutional processes.