# What is the objective of reasoning with reinforcement learning?

arXiv: https://arxiv.org/abs/2510.13651 · HTML: https://mlsanity.com/paper/2510.13651/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 3 of 3)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce](https://mlsanity.com/paper/2504.11343/) (2025)
- [BNPO: Beta Normalization Policy Optimization](https://mlsanity.com/paper/2506.02864/) (2025)

## Built on by (most cited first, 18 of 18)

- [Maximum Likelihood Reinforcement Learning](https://mlsanity.com/paper/2602.02710/) (2026)
- [Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic](https://mlsanity.com/paper/2603.01162/) (2026)
- [Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients](https://mlsanity.com/paper/2510.23049/) (2025)
- [Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training](https://mlsanity.com/paper/2602.21189/) (2026)
- [Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2602.11128/) (2026)
- [High-Dimensional Statistics: Reflections on Progress and Open Problems](https://mlsanity.com/paper/2605.05076/) (2026)
- [Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning](https://mlsanity.com/paper/2605.06241/) (2026)
- [Tail-Likelihood Reinforcement Learning](https://mlsanity.com/paper/2609.02987/) (2026)
- [POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation](https://mlsanity.com/paper/2604.24171/) (2026)
- [ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization](https://mlsanity.com/paper/2605.12667/) (2026)
- [BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning](https://mlsanity.com/paper/2605.27293/) (2026)
- [RL2ML: Finite-Rollout Surrogate Objectives from Reinforcement Learning to Maximum Likelihood](https://mlsanity.com/paper/2605.30154/) (2026)
- [SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimation](https://mlsanity.com/paper/2608.09271/) (2026)
- [CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning](https://mlsanity.com/paper/2605.24331/) (2026)
- [Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL](https://mlsanity.com/paper/2607.01490/) (2026)
- [SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning](https://mlsanity.com/paper/2609.36552/) (2026)
- [Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO](https://mlsanity.com/paper/2609.38018/) (2026)
- [Balancing Reference Guidance and Free Generation in Trajectory Rollouts for Reasoning RL](https://mlsanity.com/paper/2610.11128/) (2026)

## Abstract (arXiv)

We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root.