# Can Language Models Use Forecasting Strategies?

arXiv: https://arxiv.org/abs/2406.04446 · HTML: https://mlsanity.com/paper/2406.04446/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting](https://mlsanity.com/paper/2210.08964/) (2022)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [Spatial-Temporal Large Language Model for Traffic Prediction](https://mlsanity.com/paper/2401.10134/) (2024)
- [Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy](https://mlsanity.com/paper/2402.19379/) (2024)
- [Forecasting Future World Events with Neural Networks](https://mlsanity.com/paper/2206.15474/) (2022)
- [The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges](https://mlsanity.com/paper/2304.05351/) (2023)
- [Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament](https://mlsanity.com/paper/2310.13014/) (2023)
- [AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy](https://mlsanity.com/paper/2402.07862/) (2024)
- [Humans vs Large Language Models: Judgmental Forecasting in an Era of Advanced AI](https://mlsanity.com/paper/2312.06941/) (2023)

## Built on by (most cited first, 13 of 13)

- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [OpenEP: Open-Ended Future Event Prediction](https://mlsanity.com/paper/2408.06578/) (2024)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [PROPHET: An Inferable Future Forecasting Benchmark with Causal Intervened Likelihood Estimation](https://mlsanity.com/paper/2504.01509/) (2025)
- [Reasoning and Tools for Human-Level Forecasting](https://mlsanity.com/paper/2408.12036/) (2024)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [Simulated Ignorance Fails: A Systematic Study of LLM Behaviors on Forecasting Problems Before Model Knowledge Cutoff](https://mlsanity.com/paper/2601.13717/) (2026)
- [Large Language Models for Predictive Analysis: How Far Are They?](https://mlsanity.com/paper/2505.17149/) (2025)
- [From Prediction to Foresight: The Role of AI in Designing Responsible Futures](https://mlsanity.com/paper/2511.21570/) (2025)
- [Verifiable Rewards for Calibrated Probabilistic Forecasting](https://mlsanity.com/paper/2607.00164/) (2026)
- [LLM-SoccerArena: Benchmarking LLMs on Real-World Predictions in Sports](https://mlsanity.com/paper/2607.24573/) (2026)
- [Global Merger-Arbitrage Forecasting with Language Models](https://mlsanity.com/paper/2607.09921/) (2026)
- [Two Heads Are Better Than One: Aggregating Weaker LLMs for Better Forecasts](https://mlsanity.com/paper/2609.33257/) (2026)

## Abstract (arXiv)

Advances in deep learning systems have allowed large models to match or surpass human accuracy on a number of skills such as image classification, basic programming, and standardized test taking. As the performance of the most capable models begin to saturate on tasks where humans already achieve high accuracy, it becomes necessary to benchmark models on increasingly complex abilities. One such task is forecasting the future outcome of events. In this work we describe experiments using a novel dataset of real world events and associated human predictions, an evaluation metric to measure forecasting ability, and the accuracy of a number of different LLM based forecasting designs on the provided dataset. Additionally, we analyze the performance of the LLM forecasters against human predictions and find that models still struggle to make accurate predictions about the future. Our follow-up experiments indicate this is likely due to models' tendency to guess that most events are unlikely to occur (which tends to be true for many prediction datasets, but does not reflect actual forecasting abilities). We reflect on next steps for developing a systematic and reliable approach to studying LLM forecasting.