mlsanityData to 7 Oct 2026

Scaling Open-Ended Reasoning to Predict the Future

arXiv 2512.25070 ↗First posted 31 Dec 2025cs.LG cs.CLcited by 19

On the map: LLM reasoning → Time series reasoning

Abstract (arXiv)

High-stakes decision making involves reasoning under uncertainty about the future. In this work, we train language models to make predictions on open-ended forecasting questions. To scale up training data, we synthesize novel forecasting questions from global events reported in daily news, using a fully automated, careful curation recipe. We train the Qwen3 thinking models on our dataset, OpenForesight. To prevent leakage of future information during training and evaluation, we use an offline news corpus, both for data generation and retrieval in our forecasting system. Guided by a small validation set, we show the benefits of retrieval, and an improved reward function for reinforcement learning (RL). Once we obtain our final forecasting system, we perform held-out testing between May to August 2025. Our specialized model, OpenForecaster 8B, matches much larger proprietary models, with our training improving the accuracy, calibration, and consistency of predictions. We find calibration improvements from forecasting training generalize across popular benchmarks. We open-source all our models, code, and data to make research on language model forecasting broadly accessible.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Time series reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 91 references that are in the map. First posted on arXiv 31 Dec 2025.

Builds on

22 of 91 references.

References · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  6. 2017Decoupled Weight Decay Regularizationcited by 1,722
  7. 2025Humanity's Last Examcited by 467
  8. 2024Measuring short-form factuality in large language modelscited by 345
  9. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  10. 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
  11. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
  12. 2022Forecasting Future World Events with Neural Networkscited by 63
10 more
  1. 2025FutureX: An Advanced Live Benchmark for LLM Agents in Future Predictioncited by 55
  2. 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 45
  3. 2020ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Datacited by 43
  4. 2025Answer Matching Outperforms Multiple Choice for Language Model Evaluationcited by 37
  5. 2024Consistency Checks for Language Model Forecasterscited by 27
  6. 2025Pitfalls in Evaluating Language Model Forecasterscited by 23
  7. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 21
  8. 2024Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oraclecited by 19
  9. 2025LLMs Can Teach Themselves to Better Predict the Futurecited by 13
  10. 2024OpenEP: Open-Ended Future Event Predictioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 19.

19 papers.

Sort:

By area:

  1. 2026Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefscited by 9
  2. 2026KellyBench: A Benchmark for Long-Horizon Sequential Decision Makingcited by 3
  3. 2026Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecastingcited by 3
  4. 2026Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theoremscited by 3
  5. 2026OracleProto: A Reproducible Framework for Benchmarking LLM Native Forecasting via Knowledge Cutoff and Temporal Maskingcited by 2
  6. 2026Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluationcited by 2
  7. 2026Agentic Time Machine as an Infrastructure for Future-Event Forecastingcited by 2
  8. 2026What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulnesscited by 2
  9. 2026YC Bench: a Live Benchmark for Forecasting Startup Outperformance in Y Combinator Batchescited by 1
  10. 2026Harnessing Pre-Resolution Signals for Future Prediction Agentscited by 1
  11. 2026Reinforcement Learning over Predictive Distributions for LLM Regressioncited by 1
  12. 2026Hindcast: Replaying Prediction Markets to Evaluate LLM Forecasterscited by 1
7 more
  1. 2026QuantSightBench: Evaluating LLM Quantitative Forecasting with Prediction Intervalscited by 0
  2. 2026GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimizationcited by 0
  3. 2026Fund2Persona: A Framework for Building and Refining Financial Advisor Personas from Fund Disclosure Datacited by 0
  4. 2026Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetrycited by 0
  5. 2026From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecastingcited by 0
  6. 2026Forecast-Dojo: Replayable Environments for Benchmarking and Training LLM Forecasting Agentscited by 0
  7. 2026CHAIN: Calibrated LLM Forecasting via Causal-Temporal Hypergraph Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning and Tools for Human-Level Forecasting
  2. 2026Do Your Own Research: Learning to Forecast by Learning to Search
  3. 2026WorldReasoner: Evaluating Whether Language Model Agents Forecast Events with Valid Reasoning
  4. 2026OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning
  5. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications
  6. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts
  7. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
  8. 2025OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data