# LLMs Can Teach Themselves to Better Predict the Future

arXiv: https://arxiv.org/abs/2502.05253 · HTML: https://mlsanity.com/paper/2502.05253/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 16 of 16)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://mlsanity.com/paper/2401.01335/) (2024)
- [Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study](https://mlsanity.com/paper/2404.10719/) (2024)
- [ExpertPrompting: Instructing Large Language Models to be Distinguished Experts](https://mlsanity.com/paper/2305.14688/) (2023)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Approaching Human-Level Forecasting with Language Models](https://mlsanity.com/paper/2402.18563/) (2024)
- [From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection](https://mlsanity.com/paper/2409.17515/) (2024)
- [Calibrating Large Language Models with Sample Consistency](https://mlsanity.com/paper/2402.13904/) (2024)
- [Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy](https://mlsanity.com/paper/2402.19379/) (2024)
- [An empirical study of LLaMA3 quantization: from LLMs to MLLMs](https://mlsanity.com/paper/2404.14047/) (2024)
- [AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrieval](https://mlsanity.com/paper/2310.01880/) (2023)

## Built on by (most cited first, 14 of 14)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Outcome-based Reinforcement Learning to Predict the Future](https://mlsanity.com/paper/2505.17989/) (2025)
- [Scaling Open-Ended Reasoning to Predict the Future](https://mlsanity.com/paper/2512.25070/) (2025)
- [AIA Forecaster: Technical Report](https://mlsanity.com/paper/2511.07678/) (2025)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [Diversity is the Strength of the AI Crowd](https://mlsanity.com/paper/2606.29661/) (2026)
- [Verifiable Rewards for Calibrated Probabilistic Forecasting](https://mlsanity.com/paper/2607.00164/) (2026)
- [PolyGnosis 2.0: Enhancing LLM Reasoning via Agentic Harness Engineering for Polymarket and OSINT Insight Extraction](https://mlsanity.com/paper/2605.25958/) (2026)
- [StakeBench: Evaluating Language Understanding Grounded in Market Commitment](https://mlsanity.com/paper/2605.26074/) (2026)
- [Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry](https://mlsanity.com/paper/2607.01661/) (2026)
- [From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecasting](https://mlsanity.com/paper/2609.05905/) (2026)
- [When Do Language Models Add Value? Relative Competence in Hybrid Event Forecasting](https://mlsanity.com/paper/2609.12101/) (2026)
- [Can LLMs Predict the Future? A Brier Score Analysis of Prediction Markets](https://mlsanity.com/paper/2609.32885/) (2026)
- [CruxBench: A Benchmark of Information Discovery](https://mlsanity.com/paper/2609.35879/) (2026)

## Abstract (arXiv)

We present an outcome-driven fine-tuning framework that enhances the forecasting capabilities of large language models (LLMs) without relying on human-curated reasoning samples. Our method leverages model self-play to generate pairs of diverse reasoning trajectories and probabilistic forecasts for a set of diverse questions that resolve after the models' knowledge cutoff date. We then rank pairs of these reasoning traces by their distance to the actual outcomes before fine-tuning the model via Direct Preference Optimization (DPO). On a separate test set, our approach increases prediction accuracy of Phi-4 14B and DeepSeek-R1 14B by between 7--10% over a base model and a DPO fine-tuned control model with randomized labels, bringing them on par with forecasting capabilities of much larger frontier models like GPT-4o.