# Outcome-based Reinforcement Learning to Predict the Future

arXiv: https://arxiv.org/abs/2505.17989 · HTML: https://mlsanity.com/paper/2505.17989/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 15 of 15)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [Absolute Zero: Reinforced Self-play Reasoning with Zero Data](https://mlsanity.com/paper/2505.03335/) (2025)
- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models](https://mlsanity.com/paper/2310.10505/) (2023)
- [Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains](https://mlsanity.com/paper/2503.23829/) (2025)
- [RLTF: Reinforcement Learning from Unit Test Feedback](https://mlsanity.com/paper/2307.04349/) (2023)
- [1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training](https://mlsanity.com/paper/2503.19633/) (2025)
- [Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning](https://mlsanity.com/paper/2504.13941/) (2025)
- [Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning](https://mlsanity.com/paper/2412.11120/) (2024)
- [LLMs Can Teach Themselves to Better Predict the Future](https://mlsanity.com/paper/2502.05253/) (2025)

## Built on by (most cited first, 22 of 22)

- [Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty](https://mlsanity.com/paper/2507.16806/) (2025)
- [Scaling Open-Ended Reasoning to Predict the Future](https://mlsanity.com/paper/2512.25070/) (2025)
- [Future-as-Label: Scalable Supervision from Real-World Outcomes](https://mlsanity.com/paper/2601.06336/) (2026)
- [Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs](https://mlsanity.com/paper/2604.18576/) (2026)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models](https://mlsanity.com/paper/2603.24844/) (2026)
- [FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards](https://mlsanity.com/paper/2604.26733/) (2026)
- [OracleProto: A Reproducible Framework for Benchmarking LLM Forecasting via Knowledge Cutoffs and Temporal Masking](https://mlsanity.com/paper/2605.03762/) (2026)
- [Reinforcement Learning for LLM-based Event Forecasting](https://mlsanity.com/paper/2606.15917/) (2026)
- [Diversity is the Strength of the AI Crowd](https://mlsanity.com/paper/2606.29661/) (2026)
- [Verifiable Rewards for Calibrated Probabilistic Forecasting](https://mlsanity.com/paper/2607.00164/) (2026)
- [Aligning Language Models with Selective Prediction](https://mlsanity.com/paper/2607.03528/) (2026)
- [On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models](https://mlsanity.com/paper/2607.04332/) (2026)
- [Harnessing Pre-Resolution Signals for Future Prediction Agents](https://mlsanity.com/paper/2604.15719/) (2026)
- [Reinforcement Learning over Predictive Distributions for LLM Regression](https://mlsanity.com/paper/2605.20740/) (2026)
- [LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications](https://mlsanity.com/paper/2608.23058/) (2026)
- [PolyGnosis 2.0: Enhancing LLM Reasoning via Agentic Harness Engineering for Polymarket and OSINT Insight Extraction](https://mlsanity.com/paper/2605.25958/) (2026)
- [Global Merger-Arbitrage Forecasting with Language Models](https://mlsanity.com/paper/2607.09921/) (2026)
- [Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis](https://mlsanity.com/paper/2607.25554/) (2026)
- [From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecasting](https://mlsanity.com/paper/2609.05905/) (2026)
- [When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimation](https://mlsanity.com/paper/2609.34454/) (2026)
- [CruxBench: A Benchmark of Information Discovery](https://mlsanity.com/paper/2609.35879/) (2026)

## Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has been an effective approach for improving Large Language Models' reasoning in domains such as coding and mathematics. Here, we apply RLVR methods towards forecasting future real-world events - a challenging task for RL due to the very noisy (and delayed) outcomes involved. Using a novel dataset of recent questions from a prediction market, and accompanying relevant news headlines, we show that a compact (14B) reasoning model can be trained to match or surpass the predictive accuracy of frontier models like o1, while greatly improving probabilistic calibration. The model's performance is also practically meaningful: in a Polymarket trading simulation, we estimate that its bets would have yielded a return on investment of over 10% across all questions in the test set. We detail and compare approaches used in training our model, including augmenting our training-data with synthetic prediction questions, guardrails for learning stability, and median prediction sampling at inference-time.