mlsanityData to 14 Oct 2026

What Will Post-Training Fix? Per-Problem Gains Are Shared Across Independent RL Runs, and Existing Checkpoints Predict Them Better Than A Priori Signals

arXiv 2610.04978 ↗First posted 4 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Data selection, curricula and the evaluation of post-training recipes all assume that we can tell, before training, which problems a model will improve on. We test this assumption directly. For two base models, DeepSeek-R1-Distill-Qwen-1.5B and Qwen2.5-Math-1.5B, we evaluate eighteen post-training runs on up to 1532 competition math problems with many samples per problem, and compare signals available before training against a noise ceiling derived from the agreement between disjoint subsets of runs. Three findings hold on both base models. What post-training fixes is shared: independent runs agree on which rarely solved problems improve, with a noise ceiling of about 0.9, yet the two base models agree with each other only at rho=0.25 - the shared component belongs to the base model, not to the problem. A priori signals capture a minority of it: base pass rate, the likelihood of a correct solution, a larger model's pass rate and their combinations explain only 0.30 and 0.24 of the explainable variance in gains. Existing checkpoints are the better predictor: the per-problem gains of a single checkpoint from another family predict a new run better than every a priori signal, alone or combined (0.52 vs. 0.33 and 0.34 vs. 0.17 against the combined signals). The conclusions hold on problems from 2025-2026 competitions and when the baselines on the two sides of every comparison are estimated independently. We propose the noise ceiling as a standard companion to per-problem signals.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

33 of 39 references.

References · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  8. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  9. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  10. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  11. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  12. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 164
21 more
  1. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 97
  2. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 74
  3. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  4. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 64
  5. 2025ExGRPO: Learning to Reason from Experiencecited by 53
  6. 2025FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Modelscited by 27
  7. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 24
  8. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulationscited by 19
  9. 2026Understanding Diversity Collapse in RLVR via the Lens of Overtrainingcited by 10
  10. 2026The Unlearnability Phenomenon in RLVR for Language Modelscited by 3
  11. 2026New Skills or Sharper Primitives? A Probabilistic Perspective on the Emergence of Reasoning in RLVRcited by 2
  12. 2026TREK: Distill to Explore, Reinforce to Refinecited by 2
  13. 2026The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Contextcited by 2
  14. 2026Beyond Solvability: Task Learnability as a Static Prior for LLM RL Post-Trainingcited by 2
  15. 2026DataFlex-RL: An Evaluation Platform for RLVR Data Policiescited by 2
  16. 2026CircuitLens: Reasoning Circuits as Data Selection Signals for Reinforcement Learning with Verifiable Rewardscited by 1
  17. 2026RLVR is a Kernel, Not a Function: Statistical Inference for pass@k Crossoverscited by 1
  18. 2026Beyond Imitation: Auditing the Recoverability of Reasoning in Distilled Modelscited by 1
  19. 2026The Model Knows Another Way: Strategy Switching for Effective RLVR Explorationcited by 1
  20. 2026Solving Without Stopping: On-Policy Distillation at Small Scalecited by 1
  21. 2026Unlearnable, or Unmeasured? On the Reliability of Difficulty Labels in RLVRcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training
  2. 2024Progress or Regress? Self-Improvement Reversal in Post-training
  3. 2026Prioritized Replay for RL Post-training
  4. 2026Demystifying Reinforcement Learning Post-Training of Language Models
  5. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
  6. 2026Do Post-Training Algorithms Actually Differ? A Controlled Study Across Model Scales Uncovers Scale-Dependent Ranking Inversions
  7. 2026Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
  8. 2026The Impact of Post-training on Data Contamination