mlsanityData to 14 Oct 2026

Reinforcement Pre-Training

arXiv 2506.08007 ↗First posted 9 Jun 2025cs.CLcited by 40

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

17 of 21 references.

References · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  10. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  11. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  12. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
5 more
  1. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  2. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  3. 2025Reinforcing General Reasoning without Verifierscited by 77
  4. 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 66
  5. 2025On-Policy RL with Optimal Reward Baselinecited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 21.

40 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025A Survey on Latent Reasoningcited by 81
  4. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  5. 2025Generalist Reward Models: Found Inside Large Language Modelscited by 34
  6. 2025RLP: Reinforcement as a Pretraining Objectivecited by 25
  7. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 24
  8. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  9. 2025Reinforcement Learning on Pre-Training Datacited by 12
  10. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
  11. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelscited by 8
  12. 2025PretrainZero: Reinforcement Active Pretrainingcited by 8
28 more
  1. 2026Likelihood-Based Reward Designs for General LLM Reasoningcited by 7
  2. 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Votingcited by 7
  3. 2025Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impactscited by 6
  4. 2025Reinforcement Mid-Trainingcited by 6
  5. 2026Self-Improving Pretraining: using post-trained models to pretrain better modelscited by 6
  6. 2025Thinking Augmented Pre-trainingcited by 4
  7. 2025Learning to Reason as Action Abstractions with Scalable Mid-Training RLcited by 3
  8. 2025DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Modelcited by 3
  9. 2026Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Modelscited by 3
  10. 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
  11. 2025BOW: Training Language Models to Reason Over Plausible Next Wordscited by 2
  12. 2025Catch Your Breath: Adaptive Computation for Self-Paced Sequence Productioncited by 2
  13. 2025Refine Thought: A Test-Time Inference Method for Embedding Model Reasoningcited by 2
  14. 2026Cognitive Training for Language Models: Towards General Capabilities via Cross-Entropy Gamescited by 2
  15. 2025Diversity or Precision? A Deep Dive into Next Token Predictioncited by 1
  16. 2026PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginningcited by 1
  17. 2026Reinforced Fast Weights with Next-Sequence Predictioncited by 1
  18. 2026Towards Better RL Training Data Utilization via Second-Order Rolloutcited by 1
  19. 2026Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoningcited by 1
  20. 2026RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM trainingcited by 1
  21. 2026CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learningcited by 0
  22. 2026SemRep: Generative Code Representation Learning with Code Transformationscited by 0
  23. 2026PubSwap: Public-Data Off-Policy Coordination for Federated RLVRcited by 0
  24. 2026Likelihood scoring for continuations of mathematical text: a self-supervised benchmark with tests for shortcut vulnerabilitiescited by 0
  25. 2026MemTrain: Self-Supervised Context Memory Trainingcited by 0
  26. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimizationcited by 0
  27. 2026Is Next-Chunk Reasoning RL Really Better than SFT? Revisiting Training Strategies under no-CoT Datacited by 0
  28. 2026T5: Twin-Critic Training for Token-Level Thoughts in Reinforcement Mid-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs
  3. 2026Reinforcement Learning with Promising Tokens for Large Language Models
  4. 2026Understanding Reasoning from Pretraining to Post-Training
  5. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  6. 2025Behavior Injection: Preparing Language Models for Reinforcement Learning
  7. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
  8. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection