mlsanityData to 14 Oct 2026

SAPO: Self-Adaptive Process Optimization Makes Small Reasoners Stronger

arXiv 2601.20312 ↗First posted 28 Jan 2026cs.CLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Existing self-evolution methods overlook the influence of fine-grained reasoning steps, which leads to the reasoner-verifier gap. The computational inefficiency of Monte Carlo (MC) process supervision further exacerbates the difficulty in mitigating the gap. Motivated by the Error-Related Negativity (ERN), which the reasoner can localize error following incorrect decisions, guiding rapid adjustments, we propose a Self-Adaptive Process Optimization (SAPO) method for self-improvement in Small Language Models (SLMs). SAPO adaptively and efficiently introduces process supervision signals by actively minimizing the reasoner-verifier gap rather than relying on inefficient MC estimations. Extensive experiments demonstrate that the proposed method outperforms most existing self-evolution methods on two challenging task types: mathematics and code. Additionally, to further investigate SAPO's impact on verifier performance, this work introduces two new benchmarks for process reward models in both mathematical and coding tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 47 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

26 of 47 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Inference-Time Scaling for Generalist Reward Modeling3 Apr
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  5. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  6. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning9 Dec
  7. 2024Free Process Rewards without Process Labels2 Dec
  8. 2024Process Reward Model with Q-Value Rankings15 Oct
  9. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun
  10. 2022Solving math word problems with process- and outcome-based feedback25 Nov
  11. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 15

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  9. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  10. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  11. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  12. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
3 more
  1. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  2. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  3. 2025Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extractioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
  2. 2024Learning to Reason via Self-Iterative Process Feedback for Small Language Models
  3. 2026A Task-Centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula
  4. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  5. 2026SAHOO: Safeguarded Alignment for High-Order Optimization Objectives in Recursive Self-Improvement
  6. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
  7. 2025Process-based Self-Rewarding Language Models
  8. 2026Towards Robust Process Reward Modeling via Noise-aware Learning