mlsanityData to 7 Oct 2026

From Rollouts to Recipes: Self-Contained Post-Training for LLMs

arXiv 2609.01422 ↗First posted 1 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training large language models usually applies a single training recipe to all samples, even though the model's own rollouts reveal different sample-level learning states. We propose Self-Routing, a behavior-conditioned post-training framework that uses rollout correctness and confidence to decide how each sample should be optimized. Depending on its behavior state, a sample is routed to GRPO, on-policy self-distillation, regularization, or skipping, allowing training to adapt without external teachers, extra annotations, or additional sampling. Experiments on mathematical reasoning across Qwen3 and Qwen3.5 backbones show that Self-Routing consistently improves over uniform GRPO, uniform OPSD, fixed mixtures, and simpler routing baselines. Further analyses show that the routing distribution changes over training and reduces unnecessary updates on low-signal or already stable samples.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

31 of 42 references.

References · 31

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  12. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
19 more
  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  2. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  3. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  4. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  5. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  6. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  7. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  9. 2026Reinforcement Learning via Self-Distillationcited by 285
  10. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  11. 2023Universal Self-Consistency for Large Language Model Generationcited by 198
  12. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  13. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  14. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learningcited by 84
  15. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  16. 2026Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agentscited by 17
  17. 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillationcited by 3
  18. 2026Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scalingcited by 1
  19. 2026Self-Specialized Teachers for Domain Post-Trainingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
  2. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models
  3. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
  4. 2025Towards a Unified View of Large Language Model Post-Training
  5. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  6. 2026PRISM: A Unified Framework for Post-Training LLMs Without Verifiable Rewards
  7. 2025LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents
  8. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training