mlsanityData to 14 Oct 2026

CoScale-RL: Efficient Post-Training by Co-Scaling Data and Computation

arXiv 2601.14695 ↗First posted 21 Jan 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Training Large Reasoning Model (LRM) is usually unstable and unpredictable, especially on hard problems or weak foundation models. We found that the current post-training scaling strategy can still improve on these cases. We propose CoScale-RL, a novel scaling strategy with better data and computational efficiency. We first scale up solutions to make problems solvable. The core idea is to collect multiple solutions for each problem, rather than simply enlarging the dataset. Then, we scale up rollout computation to stabilize Reinforcement Learning. We further leverage a model merge technique called Re-distillation to sustain or even improve computational efficiency when scaling up. Our method significantly improves data and computational efficiency, with an average 3.76× accuracy improvement on four benchmarks. CoScale-RL is able to improve an LRM's ability boundary without an extensive SFT dataset. Our method provides a new scaling direction to further improve LRM's reasoning ability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 50 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

38 of 50 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacement3 Dec
  2. 2025BroRL: Scaling Reinforcement Learning via Broadened Exploration1 Oct
  3. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR19 Aug
  4. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
  5. 2025Towards Revealing the Effectiveness of Small-Scale Fine-tuning in R1-style Reinforcement Learning23 May
  6. 2025Learning to Reason under Off-Policy Guidance21 Apr
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  8. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining10 Apr
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  10. 2025LIMR: Less is More for RL Scaling17 Feb
  11. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan

Other references · 26

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  6. 2025s1: Simple test-time scalingcited by 1,341
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  9. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  10. 2018An Empirical Model of Large-Batch Trainingcited by 281
  11. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
  12. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
14 more
  1. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 114
  2. 2022On the SDEs and Scaling Rules for Adaptive Gradient Algorithmscited by 104
  3. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 101
  4. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 70
  5. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Mergingcited by 58
  6. 2023Combinatorial Optimization with Policy Adaptation using Latent Space Searchcited by 41
  7. 2024Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noisecited by 23
  8. 2024Surge Phenomenon in Optimal Learning Rate and Batch Size Scalingcited by 22
  9. 2025The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMscited by 12
  10. 2025Controlling Thinking Speed in Reasoning Modelscited by 10
  11. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelscited by 8
  12. 2025Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoningcited by 5
  13. 2025Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategiescited by 2
  14. 2025Distilling Reinforcement Learning into Single-Batch Datasetscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models
  2. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  3. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
  4. 2026Does Scaling Reinforcement Learning Really Require More Training?
  5. 2026SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoning
  6. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  7. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
  8. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection