# AIA Forecaster: Technical Report

arXiv: https://arxiv.org/abs/2511.07678 · HTML: https://mlsanity.com/paper/2511.07678/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 62 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 21 of 21)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Evaluating the World Model Implicit in a Generative Model](https://mlsanity.com/paper/2406.03689/) (2024)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection](https://mlsanity.com/paper/2409.17515/) (2024)
- [On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)](https://mlsanity.com/paper/2302.06706/) (2023)
- [Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy](https://mlsanity.com/paper/2402.19379/) (2024)
- [Forecasting Future World Events with Neural Networks](https://mlsanity.com/paper/2206.15474/) (2022)
- [Dated Data: Tracing Knowledge Cutoffs in Large Language Models](https://mlsanity.com/paper/2403.12958/) (2024)
- [ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data](https://mlsanity.com/paper/2005.00792/) (2020)
- [MIRAI: Evaluating LLM Agents for Event Forecasting](https://mlsanity.com/paper/2407.01231/) (2024)
- [Pitfalls in Evaluating Language Model Forecasters](https://mlsanity.com/paper/2506.00723/) (2025)
- [LLMs Can Teach Themselves to Better Predict the Future](https://mlsanity.com/paper/2502.05253/) (2025)
- [Are You Smarter Than a Random Expert? The Robust Aggregation of Substitutable Signals](https://mlsanity.com/paper/2111.03153/) (2021)
- [FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark](https://mlsanity.com/paper/2502.19676/) (2025)
- [PROPHET: An Inferable Future Forecasting Benchmark with Causal Intervened Likelihood Estimation](https://mlsanity.com/paper/2504.01509/) (2025)
- [Prompt Engineering Large Language Models' Forecasting Capabilities](https://mlsanity.com/paper/2506.01578/) (2025)
- [Analyzing the Role of Context in Forecasting with Large Language Models](https://mlsanity.com/paper/2501.06496/) (2025)

## Built on by (most cited first, 9 of 9)

- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting](https://mlsanity.com/paper/2605.27668/) (2026)
- [Belief Propagation in LLM World Models: Measuring Strategic Information Bias with Prediction Markets](https://mlsanity.com/paper/2607.20441/) (2026)
- [MadEvolve: Evolutionary Optimization of Trading Systems with Large Language Models](https://mlsanity.com/paper/2605.23007/) (2026)
- [Global Merger-Arbitrage Forecasting with Language Models](https://mlsanity.com/paper/2607.09921/) (2026)
- [When Should Forecasting Agents Reason? Behavioral Stress Tests for Reliability Routing](https://mlsanity.com/paper/2609.28475/) (2026)
- [Can LLMs Predict the Future? A Brier Score Analysis of Prediction Markets](https://mlsanity.com/paper/2609.32885/) (2026)
- [CruxBench: A Benchmark of Information Discovery](https://mlsanity.com/paper/2609.35879/) (2026)

## Abstract (arXiv)

This technical report describes the AIA Forecaster, a Large Language Model (LLM)-based system for judgmental forecasting using unstructured data. The AIA Forecaster approach combines three core elements: agentic search over high-quality news sources, a supervisor agent that reconciles disparate forecasts for the same event, and a set of statistical calibration techniques to counter behavioral biases in large language models. On the ForecastBench benchmark (Karger et al., 2024), the AIA Forecaster achieves performance equal to human superforecasters, surpassing prior LLM baselines. In addition to reporting on ForecastBench, we also introduce a more challenging forecasting benchmark sourced from liquid prediction markets. While the AIA Forecaster underperforms market consensus on this benchmark, an ensemble combining AIA Forecaster with market consensus outperforms consensus alone, demonstrating that our forecaster provides additive information. Our work establishes a new state of the art in AI forecasting and provides practical, transferable recommendations for future research. To the best of our knowledge, this is the first work that verifiably achieves expert-level forecasting at scale.