mlsanityData to 14 Oct 2026

What is the objective of reasoning with reinforcement learning?

arXiv 2510.13651 ↗First posted 15 Oct 2025cs.LG math.OCcited by 18

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 3 of its 7 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

3 of 7 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025BNPO: Beta Normalization Policy Optimization3 Jun
  2. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce15 Apr

Other references · 1

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 17.

18 papers.

Sort:

By area:

  1. 2026Maximum Likelihood Reinforcement Learningcited by 36
  2. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statisticcited by 21
  3. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradientscited by 11
  4. 2026Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-trainingcited by 8
  5. 2026Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewardscited by 4
  6. 2026High-Dimensional Statistics: Reflections on Progress and Open Problemscited by 4
  7. 2026Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learningcited by 3
  8. 2026Tail-Likelihood Reinforcement Learningcited by 3
  9. 2026POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generationcited by 1
  10. 2026ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimizationcited by 1
  11. 2026BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoningcited by 1
  12. 2026RL2ML: Finite-Rollout Surrogate Objectives from Reinforcement Learning to Maximum Likelihoodcited by 1
6 more
  1. 2026SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimationcited by 1
  2. 2026CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoningcited by 0
  3. 2026Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RLcited by 0
  4. 2026SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learningcited by 0
  5. 2026Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPOcited by 0
  6. 2026Balancing Reference Guidance and Free Generation in Trajectory Rollouts for Reasoning RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening
  2. 2026Transformation-Augmented GRPO for Enhancing Exploration in Reasoning of Large Language Models
  3. 2025XRPO: Pushing the limits of GRPO with Targeted Exploration and Exploitation
  4. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  5. 2025TreeRPO: Tree Relative Policy Optimization
  6. 2026Likelihood-Based Reward Designs for General LLM Reasoning
  7. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models
  8. 2026DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning