mlsanityData to 7 Oct 2026

LLMs Can Teach Themselves to Better Predict the Future

arXiv 2502.05253 ↗First posted 7 Feb 2025cs.CL cs.AIcited by 13

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

We present an outcome-driven fine-tuning framework that enhances the forecasting capabilities of large language models (LLMs) without relying on human-curated reasoning samples. Our method leverages model self-play to generate pairs of diverse reasoning trajectories and probabilistic forecasts for a set of diverse questions that resolve after the models' knowledge cutoff date. We then rank pairs of these reasoning traces by their distance to the actual outcomes before fine-tuning the model via Direct Preference Optimization (DPO). On a separate test set, our approach increases prediction accuracy of Phi-4 14B and DeepSeek-R1 14B by between 7--10% over a base model and a DPO fine-tuned control model with randomized labels, bringing them on par with forecasting capabilities of much larger frontier models like GPT-4o.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 16 of its 25 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

16 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 15

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  6. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  7. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  8. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 140
  9. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  10. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
  11. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  12. 2024Calibrating Large Language Models with Sample Consistencycited by 72
3 more
  1. 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 66
  2. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
  3. 2023AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrievalcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 8.

13 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 21
  3. 2025Scaling Open-Ended Reasoning to Predict the Futurecited by 19
  4. 2025AIA Forecaster: Technical Reportcited by 9
  5. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impactscited by 6
  6. 2026Verifiable Rewards for Calibrated Probabilistic Forecastingcited by 2
  7. 2026PolyGnosis 2.0: Enhancing LLM Reasoning via Agentic Harness Engineering for Polymarket and OSINT Insight Extractioncited by 0
  8. 2026StakeBench: Evaluating Language Understanding Grounded in Market Commitmentcited by 0
  9. 2026Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetrycited by 0
  10. 2026From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecastingcited by 0
  11. 2026Competence-Gated Pooling of Language Models and Priors for Event Forecastingcited by 0
  12. 2026Can LLMs Predict the Future? A Brier Score Analysis of Prediction Marketscited by 0
1 more
  1. 2026CruxBench: A Benchmark of Information Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications
  2. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  3. 2025Post-Training LLMs as Better Decision-Making Agents: A Regret-Minimization Approach
  4. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMs
  5. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models
  6. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
  7. 2025On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalization
  8. 2022Large Language Models Can Self-Improve