# Forecasting Future World Events with Neural Networks

arXiv: https://arxiv.org/abs/2206.15474 · HTML: https://mlsanity.com/paper/2206.15474/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 75 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Posterior calibration and exploratory analysis for natural language processing models](https://mlsanity.com/paper/1508.05154/) (2015)
- [X-Risk Analysis for AI Research](https://mlsanity.com/paper/2206.05862/) (2022)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [Individual Calibration with Randomized Forecasting](https://mlsanity.com/paper/2006.10288/) (2020)
- [ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data](https://mlsanity.com/paper/2005.00792/) (2020)

## Built on by (most cited first, 30 of 63)

- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?](https://mlsanity.com/paper/2407.21792/) (2024)
- [Evaluating Superhuman Models with Consistency Checks](https://mlsanity.com/paper/2306.09983/) (2023)
- [LLM-as-a-Prophet: Understanding Predictive Intelligence with Prophet Arena](https://mlsanity.com/paper/2510.17638/) (2025)
- [Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning](https://mlsanity.com/paper/2305.10613/) (2023)
- [Consistency Checks for Language Model Forecasters](https://mlsanity.com/paper/2412.18544/) (2024)
- [Is Your LLM Outdated? A Deep Look at Temporal Generalization](https://mlsanity.com/paper/2405.08460/) (2024)
- [Natural Selection Favors AIs over Humans](https://mlsanity.com/paper/2303.16200/) (2023)
- [Pitfalls in Evaluating Language Model Forecasters](https://mlsanity.com/paper/2506.00723/) (2025)
- [Mitigating Temporal Misalignment by Discarding Outdated Facts](https://mlsanity.com/paper/2305.14824/) (2023)
- [Scaling Open-Ended Reasoning to Predict the Future](https://mlsanity.com/paper/2512.25070/) (2025)
- [Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle](https://mlsanity.com/paper/2411.08324/) (2024)
- [AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrieval](https://mlsanity.com/paper/2310.01880/) (2023)
- [Predicting Empirical AI Research Outcomes with Language Models](https://mlsanity.com/paper/2506.00794/) (2025)
- [Can Language Models Use Forecasting Strategies?](https://mlsanity.com/paper/2406.04446/) (2024)
- [Towards the Scalable Evaluation of Cooperativeness in Language Models](https://mlsanity.com/paper/2303.13360/) (2023)
- [PolyBench: Benchmarking LLM Forecasting and Trading Capabilities on Live Prediction Market Data](https://mlsanity.com/paper/2604.14199/) (2026)
- [Conditioning Predictive Models: Risks and Strategies](https://mlsanity.com/paper/2302.00805/) (2023)
- [Humans vs Large Language Models: Judgmental Forecasting in an Era of Advanced AI](https://mlsanity.com/paper/2312.06941/) (2023)
- [OpenEP: Open-Ended Future Event Prediction](https://mlsanity.com/paper/2408.06578/) (2024)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [Event-level Knowledge Editing](https://mlsanity.com/paper/2402.13093/) (2024)
- [AIA Forecaster: Technical Report](https://mlsanity.com/paper/2511.07678/) (2025)
- [FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark](https://mlsanity.com/paper/2502.19676/) (2025)
- [Reasoning and Tools for Human-Level Forecasting](https://mlsanity.com/paper/2408.12036/) (2024)
- [Superintelligence Strategy: Expert Version](https://mlsanity.com/paper/2503.05628/) (2025)
- [Drafting Event Schemas using Language Models](https://mlsanity.com/paper/2305.14847/) (2023)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [Do Large Language Models Know What They Don't Know? Kalshibench: A New Benchmark for Evaluating Epistemic Calibration via Prediction Markets](https://mlsanity.com/paper/2512.16030/) (2025)

## Abstract (arXiv)

Forecasting future world events is a challenging but valuable task. Forecasts of climate, geopolitical conflict, pandemics and economic indicators help shape policy and decision making. In these domains, the judgment of expert humans contributes to the best forecasts. Given advances in language modeling, can these forecasts be automated? To this end, we introduce Autocast, a dataset containing thousands of forecasting questions and an accompanying news corpus. Questions are taken from forecasting tournaments, ensuring high quality, real-world importance, and diversity. The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts (avoiding leakage from the future). Motivated by the difficulty of forecasting numbers across orders of magnitude (e.g. global cases of COVID-19 in 2022), we also curate IntervalQA, a dataset of numerical questions and metrics for calibration. We test language models on our forecasting task and find that performance is far below a human expert baseline. However, performance improves with increased model size and incorporation of relevant information from the news corpus. In sum, Autocast poses a novel challenge for large language models and improved performance could bring large practical benefits.