mlsanityData to 14 Oct 2026

Outcome-based Reinforcement Learning to Predict the Future

arXiv 2505.17989 ↗First posted 23 May 2025cs.LG cs.AIcited by 22

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has been an effective approach for improving Large Language Models' reasoning in domains such as coding and mathematics. Here, we apply RLVR methods towards forecasting future real-world events - a challenging task for RL due to the very noisy (and delayed) outcomes involved. Using a novel dataset of recent questions from a prediction market, and accompanying relevant news headlines, we show that a compact (14B) reasoning model can be trained to match or surpass the predictive accuracy of frontier models like o1, while greatly improving probabilistic calibration. The model's performance is also practically meaningful: in a Polymarket trading simulation, we estimate that its bets would have yielded a return on investment of over 10% across all questions in the test set. We detail and compare approaches used in training our model, including augmenting our training-data with synthetic prediction questions, guardrails for learning stability, and median prediction sampling at inference-time.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 20 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

15 of 20 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  2. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning15 Apr
  3. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains31 Mar
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  5. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 7

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 187
  4. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  5. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
  6. 2024Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learningcited by 23
  7. 2025LLMs Can Teach Themselves to Better Predict the Futurecited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 19.

22 papers.

Sort:

By area:

  1. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 117
  2. 2025Scaling Open-Ended Reasoning to Predict the Futurecited by 21
  3. 2026Future-as-Label: Scalable Supervision from Real-World Outcomescited by 11
  4. 2026Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefscited by 10
  5. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impactscited by 6
  6. 2026Reaching Beyond the Mode: RL for Distributional Reasoning in Language Modelscited by 6
  7. 2026FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewardscited by 2
  8. 2026OracleProto: A Reproducible Framework for Benchmarking LLM Forecasting via Knowledge Cutoffs and Temporal Maskingcited by 2
  9. 2026Reinforcement Learning for LLM-based Event Forecastingcited by 2
  10. 2026Diversity is the Strength of the AI Crowdcited by 2
  11. 2026Verifiable Rewards for Calibrated Probabilistic Forecastingcited by 2
  12. 2026Aligning Language Models with Selective Predictioncited by 2
10 more
  1. 2026On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language modelscited by 2
  2. 2026Harnessing Pre-Resolution Signals for Future Prediction Agentscited by 1
  3. 2026Reinforcement Learning over Predictive Distributions for LLM Regressioncited by 1
  4. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 1
  5. 2026PolyGnosis 2.0: Enhancing LLM Reasoning via Agentic Harness Engineering for Polymarket and OSINT Insight Extractioncited by 0
  6. 2026Global Merger-Arbitrage Forecasting with Language Modelscited by 0
  7. 2026Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesiscited by 0
  8. 2026From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecastingcited by 0
  9. 2026When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimationcited by 0
  10. 2026CruxBench: A Benchmark of Information Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  2. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  3. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
  4. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  5. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  6. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  7. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
  8. 2025RLVR-World: Training World Models with Reinforcement Learning