mlsanityData to 13 Oct 2026

Step-GRPO: Internalizing Dynamic Early Exit for Efficient Reasoning

arXiv 2604.16890 ↗First posted 18 Apr 2026cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large reasoning models that use long chain-of-thought excel at problem-solving yet waste compute on redundant checks. Curbing this overthinking is hard: training-time length penalties can cripple ability, while inference-time early-exit adds system overhead. To bridge this gap, we propose Step-GRPO, a novel post-training framework that internalizes dynamic early-exit capabilities directly into the model. Step-GRPO shifts the optimization objective from raw tokens to semantic steps by utilizing linguistic markers to structure reasoning. We introduce a Dynamic Truncated Rollout mechanism that exposes the model to concise high-confidence trajectories during exploration, synergized with a Step-Aware Relative Reward that dynamically penalizes redundancy based on group-level baselines. Extensive experiments across three model sizes on diverse benchmarks demonstrate that Step-GRPO achieves a superior accuracy-efficiency trade-off. On Qwen3-8B, our method reduces token consumption by 32.0% compared to the vanilla model while avoiding the accuracy degradation observed in traditional length-penalty methods.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 40 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

27 of 40 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Entropy-Guided Reasoning Compression18 Nov
  2. 2025Lexical Hints of Accuracy in LLM Reasoning Chains19 Aug
  3. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy5 Aug
  4. 2025Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards23 May
  5. 2025Dynamic Early Exit in Reasoning Models22 Apr
  6. 2025Reasoning Models Can Be Effective Without Thinking14 Apr
  7. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  8. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  4. 2025Qwen3 Technical Reportcited by 8,510
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,247
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,270
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,729
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,450
  9. 2024OpenAI o1 System Cardcited by 1,233
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,201
  11. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 548
  12. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 292
6 more
  1. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 162
  2. 2023Verbosity Bias in Preference Labeling by Large Language Modelscited by 90
  3. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  4. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Modelscited by 51
  5. 2025Stable Reinforcement Learning for Efficient Reasoningcited by 27
  6. 2025Control-R: Towards controllable test-time scalingcited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiencycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stabilizing Efficient Reasoning with Step-Level Advantage Selection
  2. 2025Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models
  3. 2026TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning
  4. 2026Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models
  5. 2025Optimizing Length Compression in Large Reasoning Models
  6. 2026NEAT: Neuron-Based Early Exit for Large Reasoning Models
  7. 2026Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning
  8. 2026APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards