mlsanityData to 7 Oct 2026

BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning

arXiv 2605.27293 ↗First posted 26 May 2026cs.LG stat.MLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existing algorithms face a tradeoff between computational efficiency and sample efficiency in value estimation and policy learning. We introduce BASIS, a critic-free post-training algorithm designed to address this tradeoff. At each online training step, BASIS samples only one rollout per prompt, but leverages rich information across prompts in the entire batch to improve value function estimation. Our experiments demonstrate that BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++, a representative single-rollout baseline, and achieves lower MSE with one rollout than group mean estimators with 8 rollouts. This improvement in value estimation translates to better policy optimization: using substantially less training time, BASIS achieves performance close to multi-rollout GRPO-type baselines and often outperforms single-rollout REINFORCE-type baselines.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 59 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

46 of 59 references.

Closest ancestors · 29

Same direction, newest first

  1. 2026Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning30 Apr
  2. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic1 Mar
  3. 2026On the Emergence of Implicit Curriculum in RLVR Learning Dynamics16 Feb
  4. 2026EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization5 Feb
  5. 2026Your Group-Relative Advantage Is Biased13 Jan
  6. 2025Shrinking the Variance: Shrinkage Baselines for Reinforcement Learning with Verifiable Rewards5 Nov
  7. 2025What is the objective of reasoning with reinforcement learning?15 Oct
  8. 2025ExGRPO: Learning to Reason from Experience2 Oct
  9. 2025Single-stream Policy Optimization16 Sep
  10. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning13 Aug
  11. 2025Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization11 Aug
  12. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback4 Aug
17 more
  1. 2025Reasoning with Exploration: An Entropy Perspective17 Jun
  2. 2025On-Policy RL with Optimal Reward Baseline29 May
  3. 2025Accelerating RL for LLM Reasoning with Optimal Advantage Regression27 May
  4. 2025Stable Reinforcement Learning for Efficient Reasoning23 May
  5. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization18 May
  6. 2025Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoning12 May
  7. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning18 Apr
  8. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce15 Apr
  9. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models13 Apr
  10. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning3 Apr
  11. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models28 Mar
  12. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  13. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  14. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  15. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  16. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  17. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  11. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  12. 2025Group Sequence Policy Optimizationcited by 788
5 more
  1. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  2. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  3. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  4. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82
  5. 2025What is the Alignment Objective of GRPO?cited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SPO++: Stream-Aligned Policy Optimization for Asynchronous Agentic RLcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
  2. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
  3. 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
  4. 2026R2PO: Decoupling Rollout and Inference Policies for LLM Reasoning
  5. 2026Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training
  6. 2026How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
  7. 2026Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning
  8. 2025Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning